Martin Meltzer

Dr. Martin I. Meltzer

Q: This is Sam Robson here with Dr. Martin Meltzer. Today's date is February 23rd, 2016, and we're here in the audio recording studio at CDC's [United States Centers for Disease Control and Prevention] Roybal Campus. I'm interviewing Martin as part of the [CDC] Ebola [Response] Oral History Project and we'll be discussing his life, his career, and especially digging into his work as part of the CDC's response to the 2014 Ebola epidemic.

So Martin, for the record, could you please state your full name and your current position with CDC?

MELTZER: Sure. Thank you, Sam. My name is Martin Isaac Meltzer. I'm a senior health economist and distinguished consultant in the Division of Preparedness and Emerging Infections within the National Center of Emerging and Zoonotic Infectious Diseases, and I head up a small health economics and modeling unit within that division.

Q: Great, thank you. Can you tell me when and where you were born?

MELTZER: I was born in what is now Harare, Zimbabwe. When I was born, it was 00:01:00actually called Salisbury of Southern Rhodesia, in 1958, so I'm coming up to my fifty-eighth birthday.

Q: Congratulations. [laughs] Tell me a bit about growing up in Southern Rhodesia.

MELTZER: Well, it was a very, very different childhood than perhaps most Americans might grow [up with]. We're Colonial British, and of course in 1965, Southern Rhodesia broke with Britain, the United Kingdom, and set up a rebel colony of Rhodesia. That's where really my formative years was in the country of Rhodesia, which had sanctions. The borders were closed, it was not easy to travel, news and books and films and all that were very difficult to come by and carefully edited and censored. So it was a very claustrophobic upbringing. We did not have rapid and easy access as one might expect nowadays with the 00:02:00internet, and as I said, there was a layer of censorship as well. So it was more confined in terms of access to knowledge and thinking and outside ideas than perhaps people elsewhere might be experiencing.

Q: What did your parents do?

MELTZER: My parents were both pharmacists. People often ask me were they farmers, being sort of colonial-type existence in Southern Africa, but they were actually pharmacists. My mother, her parents, both of them were pharmacists. So, I come from a long line of pharmacists, not farmers. And both my brother and I--I have one sibling--both decline to go anywhere near medicine, and particularly pharmacy. He's a lawyer, I'm an economist. Nothing to do with medicine.

Q: Is that completely tongue-in-cheek?

MELTZER: No, no. It was just neither of us, I think, have particular aptitude and certainly seen both our parents--we weren't particularly excited in working with the demands of treating clinical cases of medicine. Neither of us were 00:03:00particularly interested, having seen our parents close up. Dinner conversations at our house were always about treatment of this patient or that patient, difficulties of getting drugs. In Rhodesia, not all the medicines were readily available at times. So it just seemed very intense, things we were not particularly interested in.

[break]

Q: So, what did catch your interest when you were growing up?

MELTZER: One of the formative things in terms of how one's interests grow, is that I went to a high school that was--I don't know how to explain it in other terms, but was selected, was focused on engineering and sciences, which I enjoyed a lot. But I also learned while I was in the high school that as much as I might enjoy mathematics and physics and things like that, I also enjoyed the arts, and it's actually very difficult to find, even in high schools, programs 00:04:00that combine the sciences and the arts. You either did one path or another and under the British-style system of education which we followed, it literally was you did one path or the other and it was ill-equipped most of the time to deal with people that wanted a hybrid. But it was a small school relatively speaking. It still exists. And I did learn enough of both so that I knew that whatever I wanted to do, it was going to have to have combinations and, in fact, after high school I did try to be a civil engineer and I was thinking civil engineering, oh it's great, you're going to drive bulldozers and blow up stuff and build/pour concrete. And I rapidly found that engineering at the university was actually--I thought kind of boring. Lots of classroom stuff. The most boring book I've ever read was about how concrete sets. Terribly important for civil engineers no doubt, but certainly not anywhere close to the idea of communicating with people 00:05:00or anything like that. And I came back--this was a university overseas in Israel actually--I came back and I really said I don't want to go back. Civil engineering, this very science-based, engineering-based, is not me. I want to do something more.

A friend of the family who deals with the psychological assessments for companies for positioning people in the correct slot in business as it were sat down with me one Sunday afternoon and he went through an attitude test--not an aptitude test that most people take. "You just have to assume that whatever we find that you like to do you can do at least in some measure, we'll worry about that later, but let's find out," he said, "what you'd like to do." So we did the usual, those tests [that ask questions such as] do you want to be an outdoor person, a forest ranger, or do you want to sit inside and do work, all those type of questions. At the end he picked up the local prospectus--and by this 00:06:00time Rhodesia had become Zimbabwe--it was the prospectus, the catalog for the local university of Zimbabwe. Literally paged through and came to agricultural economics and said, "Hmm, this seems to be sort of the combination of things that you want." I said, "Well, what is that?" He said, "I don't know. Why don't you go talk to somebody at the university?" I went to talk to the then-dean of the agricultural college and also chair of the department of agricultural economics, and he said, "Sure, you sound like a good fit." I said, "Well, okay." It just happened to be one of those happy circumstances that I found the discipline that I--that fit me so well because things like applied economics, which is what it really is, is a mixture of science and taking in the art of understanding how people value, in other words, the economics and that type of thing. And it was purely by accident. It's not that somewhere I took in high school a class in economics, which I didn't, and I said, oh, I like this. I 00:07:00found it by accident, and while I was there, I got bitten by the research bug.

What happened was that I got a small summer grant, internship at an international research agricultural station in Nigeria, and said, this is sort of exciting for a young undergraduate to go to another country, spend six, eight weeks looking around, saw how people did research. I said, oh, this sounds interesting. I went back and talked to the dean and I said, "I really want to do research, can I do a master's and/or a PhD here or something like that?" He said, "Well, I think it's a good idea for you to do graduate work but the University of Zimbabwe is small and we really don't have the staff and capacity to manage that many graduate students." He said, "I advise you to do a graduate degree overseas." I said, "I can't afford that. Where are you thinking? Britain perhaps? Australia? Canada?" Thinking of commonwealth systems. He said, "No, no, 00:08:00no. The place to do graduate work is the US. They are the world leaders in graduate studies." I said, "I can't afford it." He said, "What you do is you go there, you perhaps pay one semester of your own tuition. Because of your background and your training here in the British-style system, you'll stomp over most of the Americans in math and they'll give you this thing called a teaching assistantship." I said, "Are you sure?" He said, "Yeah, because that's what I did."

So I applied, came across. I got married just before we came across, landed in upstate New York on January one--January two, sorry. Looked out the window and said, what's that white stuff on the ground? No idea. Had never seen snow before really on the ground. Seen pictures of it obviously. Went to Cornell University and it worked out exactly as he said. Paid for my first semester, did really well in math courses which were good quality, no two ways about it, and they 00:09:00gave me a teaching assistantship so essentially paid for my graduate studies. And then from there, so I had my graduate degrees and I was thinking that perhaps I would go back to Zimbabwe but my wife said, "In truth, somebody like you is not going to fit in diplomatically and delicately into the government now." Wrong sort of culture, if you will.

Q: What did she mean by that?

MELTZER: I'm white and the country is Black and they want Black people in government, not white people. Harsh, but a reality, and it would've been silly to ignore that. So she convinced me there were other ways to use my degrees to good effect. In fact, what I did was something very unusual. I went to the University of Florida to work on some projects as a research professor, non-tenure track, and I worked on projects actually in Zimbabwe and elsewhere in 00:10:00Africa that looked at the impact of tick-borne disease in livestock. So although I wasn't working for the government of Zimbabwe, I was working for projects that help the people of Zimbabwe. So you can come at certain things that if you want to help people or are desirous to use your skills to improve people's lives, there are more than one way to go about it.

And then from there, of course, people always ask me, well how did you end up at CDC, and CDC deals with human public health, not animal public health. So part of that answer is that after about five years at the University of Florida, the grants ran out and I had a choice of either trying to find new grants or doing something else, and at the same time at CDC they were recruiting for the first ever a class of prevention effectiveness post-doctorial fellows, and that probably takes a little bit of explaining.

In the eighties, as you know, healthcare costs in the US began to rocket, mostly 00:11:00because of a combination of more and better drugs but also more expensive because the regulations--what it required for licensure became evermore strict and therefore ever more expensive to license a drug. So you had more technologies, more procedures and costs of training MDs and staff, medical staff, were ever increasing, so healthcare costs began to soar. And the directors of CDC, both Dr. [William L.] Roper and Dr. [Jeffrey P.] Koplan, realized that they, as CDC, couldn't go continually around recommending that the public and the medical profession do this and vaccinate with that, screen for this, without at least some consideration of what is the cost of all of this, what is the cost per life saved, what is the cost per case averted? So they needed some local talent. It's not that they'd actually never done any of that at CDC, they had, but it tended to be bits and pieces and nothing particularly 00:12:00well formed and no particular program. It tended to be somebody who was interested that would do some one-time research or one or two researches and drop it out. So there was no retained memory or discipline of economics within the center and so they began to--their method was to establish a fellowship and bring in people as post-doctoral and get them involved and integrated into CDC, adopted.

So we came up here in 1995. In the first class, there were five of us. All five, by the way, still remain at CDC. So that's quite a record in terms of lack of attrition, and all five have made notable contributions to public health policymaking in terms of producing results that public health decision-makers use in terms of deciding what to recommend or what path to go in terms of 00:13:00tracking down ways to improve public health. People do ask me, say, that sounds like a nice story, but you came out of agriculture economics, you'd spent--in fact both my master's and PhD and my five years at the University of Florida was in animal health. So surely, they ask, it's very difficult or very different to go from animal health to human health. My answer is always for economists, herd of cows, herd of people, same thing. Perhaps a herd of cows is easier to work with, I don't know. [laughter] Not everybody appreciates the quiet, subtle humor of that, but it is true. From a certain level, you're looking at the transmission of infectious diseases, which is what I specialize mostly in. It's how a disease is transmitted from person to person or cow to cow and then, of course, what is the consequence of said transmission? If somebody becomes ill with the disease, what happens to them? What you really have to do is 00:14:00understand, how does a disease, how does a pathogen move through a population? That's the key part. In fact, the training and the experiences I had in animal health was enormously helpful to me and made it very easy for me to integrate or change from animal to human. I got this idea that you have to pay attention to the biology and the epidemiology. How does disease spread? Can't do anything without it. Had some great mentors at the University of Florida and I really absorbed those lessons and it became much easier for me in fact because of that background to move into CDC and understand the way that people here were thinking in terms of how does a disease spread, what can we do to intervene and prevent the spread, how do we assess the effectiveness of that, what is the minimum data we need to understand the spread and the effectiveness in interventions.

Q: Can you describe some of those people who helped with that transition?

MELTZER: Well, perhaps the most influential one was Professor [Robert Andrew "Andy"] Norval who died tragically in a traffic accident just shortly before I came up here. He was actually also a fellow Zimbabwean and world famous, justly so, for investigating tick-borne diseases in cattle and spent a lot of time in Zimbabwe and South Africa studying them and then transferred in the early nineties over to the University of Florida where we worked on this program with USAID [United States Agency for International Development] and USDA [United States Department of Agriculture]-funded elements of tick-borne and other vector-borne diseases in cattle.

Q: How was he of particular help to you?

MELTZER: Well, he was very interested in the epidemiology and the biology of tick-borne diseases, and particularly, how does it move through a cattle population and what is its impact, and obviously, what the heck do you do about 00:16:00it? He really did show me all the ways that you need to measure this, and he would ask me questions like, "What's the real economic impact of disease X?" And I'd say, "Well, how many cattle get ill?" He said, "We're not sure." So we'd work on a project that would estimate that. In fact, one of the things we did was look at the--at the time was a fairly revolutionary way because most of the time when you talk about preventing tick-borne diseases in cattle, you talk about using pesticides. In fact, the most common method, you either spray it on them or you put them in a big bath. You have to do this frequently, and in Zimbabwe, it used to be the law that cattle herds had to be dipped, as it's called, washed with pesticide thirty-two times a year in the summer every week. This is expensive. The pesticide is imported. You've also then got to spend the time and energy driving cattle to the dip tank and running them through it and then driving them back. So that takes time and energy and the question is, isn't 00:17:00there some better way to do this? So one of the ways obviously is a vaccine which we're developing and actually has been, for one of the diseases, has been licensed.

But there's also the idea of using what we call endemic stability or naturally acquired immunity. In other words, if you get a disease, exposed to the disease, and you recover--hopefully you don't die from it--you then maintain a certain degree of resistance to that disease for a certain length of time. And these diseases, actually, we figured out once they're exposed, cattle are exposed as calves, they retain for their natural lifetime immunity to it. So then the question is, well, if they have the disease all the time because they're challenged constantly, they constantly have ticks on them and constantly are being exposed to the pathogen, is there still some loss of productivity? Is it not better in fact to keep the ticks off them and disease-free? And actually we 00:18:00did some rather complicated experiments that took a lot of time to show that, no, loss of productivity is so minor as to be negligible, and in fact one of the cheapest ways of controlling this disease is to ensure that the calves are exposed to these diseases very early in life, straight after they're born when they still have some natural immunity from their mother, through the mother's milk, and are exposed in that period so they don't automatically die. They survive the first challenge and then they're naturally immunized so you don't have to worry about constantly keeping them tick-free. You do want to keep the ticks off for other reasons but not so industriously and not so expensively, and you can rely on them not dying later on in life when that's their prime productivity because there suddenly is a breakdown in the system that protects them and they're suddenly exposed to a disease they've never been protected or exposed to before.

So this was quite a revolution in terms of can you make it work. We demonstrated 00:19:00it can and it did actually work for a while. So here is a lesson in the field of, boy, you can think of disease transmission, the impact on the person or cow who gets ill, and the impact of various ways of protecting the population against this disease and all the detail and data that you need to really measure it and convince people who've been using dipping, applying pesticides for over one hundred and fifty years, so you're asking people to change habits that are being placed not just because everybody thinks it's a good idea, but in fact had been enacted in law. Now you're saying, gee, you should change it. Where's your proof? Why should we do what you say? It sounds too radical, it sounds too dangerous.

By the way, there is a human analogy. You're probably too young, Son, but you might remember in the days when I grew up, there'd be chickenpox parties. There 00:20:00wasn't a chickenpox vaccine, and so what you did was parents--some kid would get chickenpox in the neighborhood and all the parents of young kids, particularly under five, would go and expose their kids deliberately, so the kids got chickenpox early when there was less chance of severe sequelae, and then they would be protected for the rest of their lives. Some diseases, of course, you don't do that because it could be too severe, but chickenpox was a good example. This is like the cattle version of the same sort of thing: expose the kids while they're young and still have a degree of immunity. And also, they are less likely to come down with more severe consequences of the disease. It's from those kinds of lessons that seem somewhat improbable. You think there's a wide barrier, but infectious diseases spread through populations, and the way they spread and the impact--if you train yourself and think of it in a certain 00:21:00manner, your lessons can be switched over rather rapidly from one population to the other.

[break]

Q: I do appreciate the science stuff of it. I like hearing about the early encounter with the science and the economics having potentially disruptive effects socially and economically, and people have to come to terms with the fact that we have this new knowledge.

MELTZER: Of course, you know who wasn't particularly enamored with our results or our suggestions.

Q: Who would that be?

MELTZER: Guess.

Q: Regarding the vaccines for the cows?

MELTZER: Well, vaccines or the idea of not using so much pesticide on them.

Q: Oh, like Monsanto?

MELTZER: Yeah. Well, not a particular company, [laughter] but companies making pesticides. We are suggesting--because they've built a small, you know, it wasn't middle of southern Africa, it's not a huge business of foreign currencies 00:22:00difficult even nowadays, you know, there's countries always struggling with how to pay for foreign stuff bought overseas. But how a small little business, you can almost bet and know what they're going to order every year, so it's stable. And here we come along saying actually you don't need to buy so much pesticide. Not popular in certain quarters. And anybody who's surprised by that reaction I think is not really thinking through all the consequences. Nothing is without consequence. It's just a question of how much and who bears it. So you've always got to deal with it, and that's a lesson that's very much applicable in human public health as well. There's always some consequences of adopting a new strategy or dealing with a particular disease in a way that hereforto hasn't been dealt with, so it's not always an everybody-wins situation. You have in 00:23:00there the very nexus of debate, which is, is this worthwhile doing? There's always some risk. The method we had was not going to guarantee that no cows would ever die from these diseases. We actually built that into the models when we looked at it. But some people might say it's unacceptable that even 2% of all cows would die. But it's still cheaper than importing the pesticides and trying to get into a system that--you import the pesticides, you've got to distribute it, you've got to have water, you've got to have the cow. It's very difficult. At one time, Rhodesia and Zimbabwe had the most extensive cattle-dipping pesticide program around, but it was not cost-free, and the question is, is it really worth it versus the alternatives?

So even when you put out the numbers, there's always debate about value, and my job even here, it's the same thing. You have a vaccine for children that has some benefits, whatever the disease is, because it prevents, protects them, but 00:24:00there are consequences. One of them, of course, is now that vaccines for children are ever more expensive because it takes longer and longer to discover them and produce them and get them licensed. Of course, talking very often about diseases that affect far fewer kids on average. This is not measles. The measles vaccine for kids is very easy, but it was developed--invented in the sixties. It's very cheap to produce and [measles] affects, if it's not vaccinated against, a lot of kids. So the number of kids you have to vaccinate to prevent one cases of measles is very few, but if you go to bacterial meningitis, which before we started vaccination programs would be about two to three thousands kids a year would get a case, and it's a very, very severe disease very often. This is a disease that can kill up to 10% of the kids who do get it, and up to 10% or 20% might have things like amputations in order to prevent the spread of 00:25:00the disease. In other words, very severe sequelae. And these are children, so it's a very emotional debate, but now you have to vaccinate thousands of kids in order to prevent one case, so what's the real value? The vaccine is not cheap.

One reason that we did come as economists to CDC was the simple fact that, for example, back in the early 1980s the current dollar equivalent of all the vaccines that CDC recommended that children have would cost about $150, $180 per kid fully vaccinated. Nowadays, it's ten times that amount. In fact for girls, because of the human papilloma virus vaccine, it's coming close to $2,000, and this is just for the vaccine doses. It says nothing about the cost of going to the doctor, the cost of a nurse's time to administer the vaccine or anything like that. Now you've got this huge increase in cost per person vaccinated. 00:26:00Somebody might say that's worthwhile, but you've always got to say, look, there's always something else you can do with that money. Is that really the tradeoff we want to do? We don't--we made a point from the day we arrived--we don't make decisions or even give an opinion as to what is worthwhile.

One of the things when I did come up to CDC, I thought well, coming up to public health I'm going to be an advocate now. I came here and quickly learned within six months I don't need to be an advocate of anything except for better science because any topic that you have, there's advocates on either side of the fence yelling and shouting and discussing and whatever. What is needed and what is often appreciated is somebody just as neutral as possible, saying here's our estimates of impact of how much it would cost to do this, what would the benefits be, what would be the side effects, the negative side effect--everything has a negative side effect--and you make the decisions. We're 00:27:00here to help you understand the numbers, answer the questions, but we are not here to make a decision.

When we first came to CDC there was some negative backlash. Economics in public health was not a standard. Some people even said to me in the hallways that they thought that economics in public health was immoral, whereupon I would say, well I think it's immoral to just spend money or recommend spending money without some appreciation of what you're doing it for, the costs and benefits. It seems logical, but some people felt very passionately that cost should not be part of public health. It's an unfortunate reality that nothing's free and everything costs and you have to balance it. But I think the key part is, if we've had some success at CDC as economists, and I think we have, is that we have kept as neutral as possible, and one of the things that we teach the new post doc 00:28:00fellows coming in is thou shalt not pick a side, thou shalt be as neutral as possible. This allows everybody to just look at the numbers, they can critique the methodology and say this was a good measurement or bad estimate or whatever but they're not saying, yes, you're just trying to promote, we're just trying to promote a number.

That is the thinking that we take even when you go into the responses, even like in Ebola. We're there to produce numbers. We're not saying you should do this or you must do that or it's heart rendering that this disease has taken hold in West Africa amongst populations that have very limited resources and are thus fragile and exposed, all of which is true, but it's not our job. There are other people to do that. The public health people do it very well. But we're there to add to their toolbox numbers and estimates that help them better understand themselves the problems of the burden of disease plus the potential impact of 00:29:00interventions, but also to better explain to other people why they should give resources to this particular intervention. Again, we should never forget at any one time there's always a demand on the public treasury. Everybody has wonderful ideas of how to spend taxpayer money. The argument we always give when you talk about childhood vaccines, you know, at some point there's a tradeoff. You could spend more money say on education. One of my favorite examples is pre-kindergarten, preschool education. It's been shown that kids who have a good preschool education do well in settling into school and get into the rhythm and get more out of education. A better educated person we know is less likely to do things like drugs and more likely to finish high school and more likely to become a productive member of the community, and that is public health in a way. 00:30:00So why not invest in education? Why must it be a vaccine? You don't always have to go to the tried and true tools. So there is a reason and a need for these types of numbers.

Q: I'm getting a good idea of the ideas that you're working with as you come to CDC and integrating economics with public health. What things are you actually doing? Are you writing reports? Are you--

MELTZER: We do a mixture. In fact, one of the things that I've done most--I believe that I've done pretty much the same sort of thing since the day I arrived over twenty years ago now, and it's hard to believe I've been here twenty years, but I don't come with my own research agenda. The best way that myself and the other economists and analysts at CDC have made an impact is that we come here to help other people analyze or answer questions that they 00:31:00have--what's the cost effectiveness, what's the value of vaccinating this group against this disease, for example. And in doing so, our research agenda, what we study is set by the people. I feel at times we really are an internal consulting agency. People come to our door with--we have a shingle outside that says Have Economists, Will Analyze, and they come with a question and we help them answer it. Now, exactly how we answer it depends very much on the question and the intended audience and over the years in fact we have developed a sort of little two-page type of outline where we have five elements in it, which the first one is: what's the question? This seems obvious. Why do you think we came to you in the first place, we have a question. But in fact, we find most of the time the questions are not very--

Q: Sorry, we need to pause.

[break]

Q: So, Martin was telling me a bit about how they work on different modeling 00:32:00jobs for various health emergencies as they come up or different questions as they come up, and so that made me remember that he is involved not just in the Ebola response but generally in like epidemic response or incident--I suppose is what you call it, incident management. So I'm wondering if you could tell us a bit about your previous experience to Ebola.

MELTZER: Actually, a lot of work we did in getting to the point where we are used routinely now in incident response is the fact of, as I was describing earlier, when we have projects, people come to our door where we're sort of internal consultants and they ask us questions. We respond, what is the question? We figure out what kind of answers do they want. What kind of graph or table would help you communicate with your primary audience? Which is the other 00:33:00thing--who's really interested? And of course, people say everybody's interested. Well, that may be true to some extent but not as interested as you might be. And who's going to take the data and perhaps use it to make a decision, for example, a minister of health; or perhaps set recommendations for an individual physician as how to treat patients; or even patients, how do I decide whether to accept a vaccine or a test or whatever is being considered? Think about your primary audience. Then once we've got all that, then we talk about the data, and that's actually the reverse of what most people think about, they come--even nowadays they come with the proverbial sack of data, throw it on my desk and say, surely you can find something of use. And we say, no, let's talk about what you want. And with that methodology of talking about the question, we've developed over the years a methodology, a protocol, a way of 00:34:00examining the question and answering it and providing news that can be used of really focusing on what I like to think of simple math models primarily built in spreadsheets. Why spreadsheets? Because nowadays most people can put a spreadsheet up on the computer and open it. They don't necessarily know how to program it and do calculations on it but they know how to maneuver around. So it's almost like a little calling card that you can pass around, and in fact over the years, particularly in response planning where we really started doing this was preparing for influenza pandemics.

One of the first things I did when I came to CDC was that the influenza then-branch, not a division, asked, could I help provide some information that would help improve the then-current US federal government pandemic plan, which I always like to characterize was about forty pages long and had lots of blank 00:35:00spaces. What the influenza branch wanted from me was some numbers that would [illustrate] the extent of burden and also the value of intervention and also some of the biggest challenges in applying those interventions. So we did, we published that paper in 1999 and the numbers have been used for pandemic planning and in fact the concepts are still being used. But one of the outcomes from all of that was that we had a set of numbers, estimates of the impact of pandemic influenza for the country and somebody said, you know, each state in America, the governor and the public health people are very interested in the impact in that state and perhaps not so interested in the impact in the state next to them or three states over because each state has a responsibility of preparing and responding to the citizens of that state, not states around them. So could I then take my model and produce something that would allow state 00:36:00health officials in each state to calculate some numbers that would help them prepare for pandemic influenza in their state only?

So we actually started producing a little program called--tool really, a model tool called FluAid which is still up on CDC's web. Initially, we distributed by CD-ROM [Compact Disc-Read Only Memory]. This was done in the days when the web was still growing. But we quickly transferred it to the web and this really helped cement, from my mind and for other people, the idea that simple tools might calculate inside, produced inside Excel-type spreadsheet programs can vastly help public health officials produce their own estimates, help them produce. A very important educational process. It's one thing if I produce a number and say here's the estimate of the number of cases in your county or 00:37:00state or country, but if they produce it and they understand what numbers have to go in, what data has to go in to produce data that's useful, then they're more likely to believe it and also more likely to understand and appreciate the quality of the data of what they know and particularly they'll really also understand what they don't know. And what they don't know is particularly important because that then says I'm going to make a decision and I'm going to assume that the percentage of people in this age group [who] get ill from this disease in this time period is X%. We don't actually have data, or we have very poor data, or we have to use data from ten years ago, but I'm going to assume that and make all my decisions based on it. They actually understand where that X% comes from and they also completely understand why that X% is important, and that's way more valuable than me writing ten textbooks or giving five hundred 00:38:00lectures because they self-taught themselves with our tools. If you want somebody to learn something, the best way to do it is have them teach themselves and that's what these tools do. We took this type of protocol, thinking, methodology into not only planning and preparing for influenza pandemic, which we did a lot of, but we also brought it when we actually began to respond to outbreaks.

Some of the earlier stuff that we did was for the SARS [severe acute respiratory syndrome] and for the anthrax, but that was less concentrated and less organized because they were not of the scale that an influenza--they were smaller scales, SARS and anthrax, than influenza pandemic. Pandemic, of course, literally "pan," 00:39:00global, certainly every state, every county in the US had cases of influenza. So we're talking scale now and when the 2009 H1N1 influenza pandemic hit, we now talked about providing estimates that were going to affect the whole population of the US. At the same time, you had a large number of people intensely interested in the numbers and you had to make those numbers, how did you calculate them, how did you assume or what did you assume in order to get to this number, what happened if you made changes in the assumptions? You've got to be as transparent as possible, so we relied, again, on the process of simple tools that we posted on the web--tools being these little simple models inside spreadsheets and constant updates and it was a success.

Initially when that started, for example, there wasn't a specific unit within 00:40:00the structure of the emergency response that said modeling. By the time 2009 ended, there was. And from then forward there's always been in the big infectious disease response a box that says modeling, which specifically means that estimates of impact of disease, potential impact of proposed interventions, duration of the epidemic with and without the responses and any other types of questions will fit in that box there, the people that man that staff, that unit, that box, inside the organization. So we have become part of the standard response because we have been able to demonstrate we're able to deliver, when circumstances allow, estimates that are news that decision-makers can use, but emphasizing again the simple tools. I think that if we had relied upon more 00:41:00complicated models, we would have less conversations with decision-makers because there would be more of a black box. They come to appreciate the simple tools, they do appreciate. People sometimes ask me, do they really understand our simple new models and I say, yes, they do. They realize that our models are not going to cover the entire--all the aspects of the disease transmission, everything, but they understand that immediately and quickly because our models are so simple and they can be used. And then, the ability to post them on the web adds to the idea that these are models that at least they might not be as complicated or cover the entire spectrum of all the risks but they increase the trust because people can always peek under the hood. The box is not black. It's as transparent as possible. You can see what's going on, and indeed because we 00:42:00post the models on the web, you can make changes. If you don't like the estimate I've put in for an important input into the model, fine. Put in your own estimate and see what happens. And just even sometimes we find that even the ability that you've allowed them to change it--they don't always make the changes, they just like the idea that if we don't trust you or they think we're pulling the wool over their eyes, they can make changes. Just the idea that you can make those changes builds a lot of trust. And then there's always people who do want to change the numbers and see what happens, which is great, which is why we put it up there.

Q: Tell me about how you got involved in the Ebola response.

MELTZER: Very simple. I remember it was August the 3rd, 2014. As I said before, I am in a unit in the Division of Preparedness and Emerging Infections whose part of our brief is to man or provide staff to emergency responses when asked. So we got an email and a phone call from the incident manager, Dr. Inger [K.] 00:43:00Damon, could we come down and help? Sure, of course, for Dr. Damon we'd do anything. And we came down and the first thing we always ask is, what's the question? What's the number one thing that you need help with, that if you knew now it will help you make decisions down the road that will influence the response? We don't come down and tell them this is what you need, though we always suspect we have a good idea. We want them to tell us exactly what happened and, of course, we always find it's become one of our mantras almost that we say the top three questions we always get initially in a response is: how many are going to die? We say, think also about hospitalizations and get the disease. How many are going to get the disease and what's the outcome from that? That's question one. The second question is, what would be the impact of these interventions that we are deploying or thinking about deploying? And the third question is, how long? When will it be over? This is all on day one mind you, 00:44:00and I always say, well, if I really knew the accurate answer to all three questions, I would be on Wall Street and in Las Vegas making a lot of money, and making that guy out in Seattle who runs Microsoft look like a pauper, if I was that good that I could be terribly accurate. But we can certainly begin to build simple models that provide some estimates and particularly illuminate for decision-makers the data that we don't have that is critical in answering those questions. So for example, said we're going to assume that X% of the population are going to become critically ill with this disease in this time period. I said, you need to know this piece of data and it's not well measured.

So we've assumed, based on data say from other outbreaks, this is what we did in Ebola. We assume that the Ebola outbreaks in West Africa were similar--the pathogen, this particular virus species was actually the same or similar enough 00:45:00to previous outbreaks that we could use data from those outbreaks and put it in the model and provide some answers. But if you think about it, that's quite an assumption that what's happening is close enough to what's happened before, and that's a very important assumption when you tell policy makers who say, well reports we're getting from the field all sounds and tastes and feels like previous outbreaks but maybe it's not. Well, if you assume it is, here are some results. If you want, we can change that. We can explore very quickly different numbers. But that's what we're basing our numbers on that we're giving you and that's what you're going to base your judgments. If you're unhappy, we can discuss and we can change. But it allowed us to very quickly illuminate that. So those are the first three questions that we really worked on. In fact, the model that we built called Ebola Response, which is available on the web, the most 00:46:00important decision we made there in model building, we said we don't just want to estimate the number of cases, which is always popular, how many, but we also want to illuminate and say what do you have to do as public health people to stop this? Because that's really why we got called in. The fact that it's bad when the epidemic was growing wasn't news. You didn't need a model for that. You got estimates or case reports every day saying it's bad. Obviously it's going to get worse before it got better. Nobody needed a modeler to tell them that. They needed a modeler to help them sort out and say, how much of a response do we need? And particularly in Ebola, how many specialized Ebola treatment units--which was the specialized hospitals that isolate patients and the staff looking after them wear a lot of personal protective gear to protect them from catching Ebola and transmitting it onwards--how many beds do we need? That's one 00:47:00of the central questions, and that second question I mentioned earlier which is, what intervention do we need? So we were able to provide estimates without intervention, what if you did absolutely nothing, and that was also important. It's actually one of the things that we've had a lot of criticism--oh, we were scaring people, it wasn't realistic. I said, no, decision-makers, if you're going to invest and provide what turned out to be billions of US taxpayer dollars, you want to know how bad could it have got if we did nothing. And in fact lots of people say--at the time we were doing the model it was getting better and better and I said, yes, because we're already pouring in hundreds of millions of dollars. When we started modeling, in fact already to that date, about $320 million had gone into the response, so we darn well hope that something was improving. But we also wanted to say, what if you did nothing else? If you just stopped the flow of funds and resources right there, what would happen? And the answer was not good. No surprise. It would continue spreading. You would have to change it. You have to assume that more resources 00:48:00were going to go in, that more patients were going to be isolated effectively either in Ebola treatment units or at least in the community where they were changing the way they looked after the sick. Not all family members were touching a sick person and most important if somebody died, because this is a very lethal pathogen as everybody knows, they didn't practice the usual burial rites of washing the body and touching the body. It went into what we call a safe burial. And did that have effect. We were able to build right from the beginning, we said, we have to build a model that shows not only the growth of cases if you do nothing but if you provide--what if 30% of the patients are put in an Ebola treatment unit, an additional 40% in the community having effective isolation and safe burials. Would that be enough to bend the curve and stop the epidemic? In fact, yes, we basically demonstrated in the model that when you get 00:49:00to about 70% of patients effectively isolated, you break effectively enough of the transmission to stop the epidemic. So we gave the decision-makers a target but also told them how.

What really was important at that time--remember this was back in August, 2014, there wasn't an effective vaccine available, there wasn't an effective treatment. So what we put in the model is interventions that we knew from prior outbreaks already worked, isolating the patients, safe burial, we knew it worked and we just said, yes, if you do enough of what we know works, you will stop this. The question is, how much is enough? So as the epidemic went on, we spent a lot of time talking to USAID, that's the Agency for International Development under the Department of State that provides aid, and particularly the Office of 00:50:00Foreign Disaster Assistance that provides a lot of resources very quickly. The question is, how much must you provide of staff and personnel in order to bend it? We spent a lot of time talking to USAID about that as well as the British equivalent of USAID, DFID [Department for International Development], essentially answering the question--they would ask us, if we provide this many staffed beds that would properly isolate patients, would it be enough to bend the curve, was the question, and we kept working back. And a lot of that was, again, conversation about what's the question? And they would say, what if we had this many beds? And we said, one of the problems isn't so much the stuff, we've solved that question quite early. OFDA [Office of US Foreign Disaster Assistance] did a fabulous job of providing stuff. It was the personnel. We said, if you think in the next month you're going to have an additional eight hundred beds properly staffed by people trained and properly equipped, that's 00:51:00fine, but where are you going to get the people to do that from zero start? And what was most important was the dialogue between us and the other plan about what was a realistic estimate of number of beds properly staffed and equipped that they could actually provide, not what they hoped. And this is a perfect example of not the complicated model but the back and forwards and getting people to understand, what is the most important thing that they don't know, and it wasn't how does the disease spread from people to people, it was how quickly could they provide people that are properly equipped and properly trained to deal with patients? That was the key thing for those groups of people who we're talking with. Again, you have an example there that as the epidemic progressed, the questions that we faced changed to meet the tone.

Towards the end of the epidemic, one of the most important questions we had was 00:52:00residual risk. We're on the downslope, the number of cases per week are dropping off. But then it became evident--something we'd sort of known before but hadn't seen much of in prior outbreaks was the risk of sexually transmitted cases of Ebola. And the question then became--well, we had a few cases that as far as we could tell were clearly transmitted by sexual contact. It's a fact of life. The question is, how often does this occur, how much residual risk is sitting out there such that you could suddenly have, say in the worst case scenario, ten or even twenty of these cases occur in one place or another and reignite the epidemic? Was that a real threat and how much of a threat? And so we worked a lot on that, particularly in the latter part of 2015, getting an assessment of how much residual risk there might be. Happy to say that we don't think there 00:53:00is--we never calculated a lot of risk and backed up by what occurred in nature, if you will, in that yes, some cases occurred but it is a rare event because if it wasn't, which we've seen with this number of cases that did occur, we would've seen a lot more cases occur from likely sexual transmission. Because there were just so many more people than we ever had before Ebola in one place. But there weren't as far as we could tell. So this risk is real but it's very, very small and we were able to provide estimates of that degree of risk in almost every month, how many people are likely to be at risk, and of course the numbers fell over time.

And that helps policymakers around the world understand the degree of risk and therefore how much they should worry about this and ameliorate fear and make it more rational. And also understand, again, we'll be making these assumptions 00:54:00regarding these tests if they provide a good estimate of risk or not and, in fact, we were of the opinion and still are that some of the test results that we were using overestimated the risk because it just measured the degree of RNA [ribonucleic acid] in survivors, male survivors' sperm, but that doesn't actually mean they can transmit the disease. So we think it was an overestimate and therefore even less reason overall to worry, and again, we were able to illustrate and say that's the number you have to watch and think a lot about what you have to worry about. So bottom line from Ebola, we were able to go in based with our past experience but also we had learned what type of questions--how to answer them and also very important, be flexible.

Q: I'm trying to get a sense also of the timeline here. The initial 00:55:00conversations start August 3rd. Is that right?

MELTZER: That's right. August 3rd, we got the infamous phone call and email. August 4th reported to CDC's Emergency Operations Center at 7:30 am and away we go.

Q: I'm interested in how it progresses. Like at what point did you decide to create the Ebola response model?

MELTZER: Again, the first three questions we often get is how many, what's the impact of the intervention and how long will this whole outbreak last with and without intervention. So within the first week that was the number one question. In fact, it originally generated from one of our CDC acting directors for the CDC teams out in Sierra Leone at the time. But before we even got the first draft out, there was senior leadership here at CDC asking exactly the same 00:56:00questions. Said, funny you should ask that, we're working on a model and we should have it to you within two days. We were working, myself and my colleague, Dr. Michael Washington, literally eighteen hours a day. I would work in the office and go home, a quick bite to eat and work late at night and working by email sending copies of the spreadsheet--again, another reason to use spreadsheets, easy to transfer between people in remote locations, copies of it as you're building it. He was checking my numbers, I was checking his calculations, we were discussing what's the best way to do this, what's the best way to do that, how can we set up the spreadsheets to make it look at least somewhat user friendly, what do we have to assume, what's the easiest way to assume, can we do without this assumption, things like that. So we were able to have the first draft in less than the first two weeks. Really the first draft was available in the first week and then we spent time going back and forth, had 00:57:00invaluable discussion with policymakers and leaders here about how they want to see the data, what questions to adjust it. So we had that done well within the first two weeks. And again, the emphasis on a simple model built in a spreadsheet [allowed it built quickly] from scratch.

Q: What was the individual's name who you were working closely with on that first draft?

MELTZER: Mike [Michael] Washington. He's part of our unit in the Division of Preparedness and Emerging Infections.

Q: Can you describe him a bit and your working relationship?

MELTZER: Yes. So, Mike has an interesting background perhaps. He's not an economist as he reminds us often. He's an operations researcher. That means he's an industrial engineer. His degrees are all in engineering, so he has an engineering's mindset that the world should be very carefully measured out and measured just so and built just so. He likes building models like that and then 00:58:00when we come to the lack of data, he finds it frustrating because in an engineer's mind everything should be carefully measured and exactly measured. But he does provide a set of analytic skills that are very valuable and in fact, over the years, I mentioned earlier on our prevention effectiveness post-doctoral fellowship, we find that we don't limit it just to economists. We opened it up years ago almost from the beginning to people who have analytic skills, and the most important part is actual flexibility of mind, are you willing to step outside the comfort of your own discipline and what you've learned and work in truly multidisciplinary teams? So even outside of a response, meeting one when you're discussing a new project, you'll often have people whose degrees in background training as an MD medical doctor, you have probably a laboratorian, somebody who's specialized in laboratory tests and analysis, very different viewpoint, very much into biology, not really 00:59:00interested in modeling. You might have somebody who's interested in public health communications. And that's who you work with. You're the only analyst of that type in that room and you're trying to get something together to answer the question.

In a response, it's a lot more heated up. The speed is way more different than your typical day-to-day non-response project. A lot more pressure as well because people will say to you, I got a phone call tomorrow at eight o'clock where we're going to discuss this, can you provide some estimates by then? Well, it's four o'clock the afternoon before and you want to start from scratch now? You know, can we--not just working ourselves. Failure, both in a response and outside a response, comes when an analyst takes a question, goes and works by themselves, literally goes to the office, closes the door, comes out a day 01:00:00later, six weeks later, six months later, it doesn't matter, and says, I have the answer for you. It doesn't work simply because they haven't communicated to people how they built the model, what the weaknesses are, what the strengths are. They've essentially built a black box and come out and said trust me, this is the answer. Well, they might trust you if they know you well enough and they've seen that your models are sufficient for their needs in the past but especially in a response, you often don't get that degree of time to build that trust because you often are working with people that have been pulled in from all over the agency and even outside the agency and they don't know your story or how you work or what you do, and there's no time. There's intense pressure. Everybody's excited, everybody's somewhat scared, so they should be initially in Ebola, and they're facing of course tremendous pressure in terms of questions: what do we do about this, what do we do about that, wait, we have to discuss this, I've got a meeting in an hour to discuss something else, can I have a 01:01:00number from you in order to use in that meeting to show people what we have to do? A lot of pressure. And so you have to have this ability to engage and be willing to discuss. Besides being an engineer with great analytic skills, Mike Washington definitely has the willingness to engage and talk to people. Of course, he has also the engineer's ability to say, why isn't everything properly measured like it should be? Why isn't everything run efficiently like it should be? But engineers are always engineers, what can I say?

Q: In this time of intense pressure, working up to eighteen-hour days, how would you describe your mood?

MELTZER: Well, of course you get very stressed, and when things don't work out very well--twenty years at CDC, the Ebola response was the single most difficult thing that I've ever worked on. Partly it was just the pressure but partly because getting data, relevant data, was very difficult. This is a combination 01:02:00of several factors. The outbreaks occurred, as you know, in West Africa in three countries, very resource poor, very few doctors, very small public health systems, not much ability to collect and analyze data. Data collection of the type that we needed was difficult. We didn't even know for sure half the time if just the basic case count was accurate. And then deciding--always there are in these cases whether it's the US, anywhere in the world, in these type of events, there's always underreporting. It's no good asking why for a variety of reasons. Fewer cases are reported than actually exist. Take that as a fact of life.

The question we want to do is get some estimate of, for every case reported, what must we multiply it in order to get an idea of the true number of cases? And that was difficult. We did some estimates. It took us over a year actually 01:03:00to get them validated. In other words, past the due date when it's useful. It's just a historical fact. "Oh yeah, we were quite accurate." But at the time we had to make some estimates trying to deal with the data, trying to deal with all the requests, trying to deal with a deal of skepticism because a lot of people had never worked really with modelers and didn't really understand how useful or more important how to interpret the answers, that these are estimates, they're not guaranteed. Yes, they might be behind what's happening on the ground--things move on the ground very fast. One of the most irritating comments I heard over and over again, "The situation is fluid." Well, that's not a surprise. Tell me what you're doing to at least measure the degree of fluidity or what you're doing to make it a lot less fluid so we have some idea so we can tell leadership, yes, this is the real picture or this is where we think it's going. 01:04:00Talking about it's fluid doesn't actually transmit--the only thing it transmits is we don't know what's going on, and it's my job to help people understand what's going on and many times it's the absence of data. So some people always push back and say your estimates are wrong. Well, actually we don't expect them to be right. There's too much unknown and too little known for us to be sure that we're right until past the point of when it's really important. It's a historical footnote when we find out how right we are, but we want to be useful and so we work under the mantra of all models are wrong but some are useful. We want to make it as useful as possible but in order to do that we must have some understanding of what's going on and to tell us the situation is fluid doesn't really help anybody. Though it might be an accurate description, what we want to know is what are you doing to measure what's going on so even if the situation is fluid we understand the degree of fluidity, where the situation is going and 01:05:00where we're expected to go next.

Q: What were people referring to when they said over and over the situation is fluid, specific to Ebola?

MELTZER: The response, where resources were going, where people, where cases were recurring, where resources to prevent future cases were going, where data are being collected. It was very difficult even to collect data. One of the difficult parts of the data was that we use cumulative case counts. In other words, if you have two cases one day and then five the next day, we don't use the two and the five, we use the seven. Some people said, oh, that's wrong, you should really focus on the two and the five. We said, yeah, but we know the data and this is one of the important parts of keeping the model simple. You can really focus on the quality of data.

There are four or five steps in getting information from the field about number of cases. One, of course, the event must occur. Two is some public health official must be told and in those countries sometimes, many times people were 01:06:00afraid to report it. And somebody has to go out and verify it's Ebola and not something else. Then they have to report to their local district office and then the district office has to report it to the central office who then enters it into the database. Now, if the time distance gap between each of those steps is roughly the same, in a very short time it sort of settles itself out and you can just assume that the daily reports then reflect it. The trouble is we knew, because we had a lot of anecdotal evidence--we spent a lot of time on the phone talking to people, how the data are collected, that there were big gaps and they changed all the time between each of those steps. In fact, one of the most infamous examples was on October the 5th in Liberia when some two thousand cases that had not been entered into the database to date were suddenly found, of which four hundred were lab confirmed cases. Wow. How did that happen? Look, these things happen. It's no good trying to blame somebody because you're not 01:07:00going to fix it. The question is, how do you deal with it? So we were using the cumulative case count but even then we had this sudden spike in cases that actually had nothing to do with how the disease was transmitted but more how the data was being transmitted along and caught up. Some people say, how did this happen, it's a failure. Well, perhaps but think again of which country you're working in, a resource-poor country with very little public health infrastructure, asking people who've never done this to do this type of information collection amongst a population not necessarily always appreciative of government coming around, not used to government actually being helpful or providing useful resources. People are scared, they don't like some of the interventions. Many times, as you know, safe burials as it was called was not appreciated because that basically changed a culture that went back hundreds of 01:08:00years of how to deal with people's burials and now you're telling them, oh, you can't touch the body, you've got to have these teams that come dressed up in this strange-looking kit and they put your departed--dear departed relative inside this body bag, you can't touch it. And sometimes there's problems, they don't even know where they took the body. Wow, no wonder you're not happy with reporting a death to Ebola to the government. The government seems to be making the situation worse rather than helping you cope with the loss of a loved one.

So all these make the collection of data difficult and you have to understand there's very little you can do to rapidly fix it to the level where you say, gee, that's worth the effort. It's always going to be problematic. So with that, go back to what's the most that you can do to help leadership when you have limited data and you've always got to accept the fact that there might be 01:09:00problems in accuracy, even when did somebody die might be inaccurate. Numbers that die in a given day or get ill. How many actually survived? What's the percentage survival rate is something that even now is being debated--how accurately is it measured, what do we know about them, what's the impact on survival rate if you go to an Ebola treatment early, say the first one or two days of when you become ill versus what if you wait five or six days. Even now I'm telling you there are people that we're analyzing the datasets but it's a big debate. It's not obvious.

So very difficult to get all that kind of data sorted out, and at the same time there's this constant pressure for what's the answer, can you help us, can you help us. It's the noise. Many times I did say, that's it, I'm resigning. And my boss would say, no, you're not. But I would've liked to, just simply because it would've taken a lot of personal stress and pressure off. I do have to say the 01:10:00part that made it worthwhile--there's a couple of things. Obviously, the rest of CDC team here, people that are committed to helping other people, this wasn't about personal gain. And two, I have the most excellent team put together, mostly the core of it was from the health economics modeling unit within the division I work, but we also called upon people throughout CDC, people I knew who understood the way we worked and I could say, come help us. We took over one of the conference rooms in Building 21 and made it our own little war room. We sat around it every day, a big, rectangular conference table, and I would assign, here's a question, this team work on it. In fact, to try and reduce the error rate, we'd have blue team, red team, we'd have the same question tackled by two different teams and then compare and say, okay, what's the error, trying to reduce error rate. When people are tired and stressed, mistakes will happen. 01:11:00You've got to try and prevent that. You can't guarantee it but having blue team, red team did actually help. Dealing with requests to meet with this taskforce or this group outside the agency, all of that takes a lot of time and energy.

One of the ones I do remember was that it was late September. A different question came up. The questions don't stay the same, they evolve to meet the situation. And the US was about to spend a lot of money very quickly and they wanted to know how much--back to this idea of, okay, we know that Ebola treatment units make a big difference, we accept that from the modeling and from past experiences. This is part of the answer, an essential part. But here's the question. How many? So, okay, that's one question. And for every bed or hundred beds you put in there, how much stuff do you need? Of course, lurking at the back of that is the accountant's question, how much is this going to cost again? 01:12:00But before you can even get to the how much cost, you must figure out the number of beds and how much stuff do they need. And never before in history had we ever needed more than forty Ebola beds--Ebola treatment unit beds in one time. Now we're suddenly talking hundreds if not in the thousands.

So, I got a call Sunday afternoon in September, I forget the exact date, I could look it up. I was walking my dog at four o'clock in the afternoon, I had actually taken a break. I get this phone call from Dr. Inger Damon, "Can you help us? We need some estimates."

"Sure, can I meet you tomorrow perhaps?"

"Well, here's the thing, there's a call tomorrow with the Department of Health and Human Services and USAID at nine o'clock in the morning and we need some estimates, can you provide some?"

"What? You want me to go from zero to full estimates?"

"Please. Yes."

So actually I called up a colleague who knows a lot about hospitals--amount of 01:13:00stuff, equipment and supplies you need, a doctor who works up in [Weill] Cornell Medical [College]--Dr. Nathaniel Hupert, who's worked with them before in several locations and has really specialized over time in planning and preparing for such emergency responses but particularly in hospital capabilities.

Literally, you're working way past midnight. I remember at three in the morning, I think I fell asleep at my desk. But we were exchanging emails and got together a report and emailed it to them and it was there on their desk by six o'clock in the morning, able to be used. But your reward for being good is that you get punished and so I thought, well, that's it, we've given them some estimates, they know how much stuff, it's a really good--we can change it, we developed a little spreadsheet over the next couple of days, they can change it if they want. If they have second thoughts about how much you need of this or that, no problem. And then they came back and said, well, the leaders found this very, 01:14:00very useful, very excited about it, it helps a lot but they want to know how much it costs. What am I? I'm not a purchasing agent. Of course, when you work in the government, you work in a very strange world. You're not allowed to purchase stuff, you've got to go through the purchasing mechanism that is federal government. And there are specialists and they deal with contracts and they get discounts but we know nothing of that. That's a very esoteric area of government. And so literally one of the things I did, I said I don't work in this area, I'm an economist, I'm not an accountant and I'm not a purchasing agent. They said, well, we did ask our purchasing agents and they looked at a list and said we don't know where to get any of this stuff, most of it, because we don't normally buy this stuff. It was a long, long list. I said, okay. So literally I said, well, we can get an initial estimate and then work over days to improve it.

So literally I said, grab even more volunteers--they don't really volunteer--I told them, come help. I said, we're going on the internet to shop for Ebola, and 01:15:00we literally went on all the websites. We said if we needed those heavy plastic aprons, find me something on the web that looks somewhat similar and get me a price per unit and we'll use that. And it was the first cut in August, actually the things that we found were too expensive. The government couldn't get them. But it allowed a first initial look and everybody could say, oh, we can get this here cheaper or you got the Rolls Royce luxury variation of this, we can do--the same job can be done with something that's a quarter of the price. That's great. But again, it was somewhat humorous amongst all the pressure of getting some numbers quickly, quickly needed because there was a budget decision to be made, budget request of how much money do we need from Congress to provide to make sure we adequately budgeted for this response. You don't want to go in there under-budgeted and you don't want to go in there with some frivolous number, you want to be able to demonstrate you've thought this out and calculated as best you can. So we started off with shopping for Ebola on the internet and then as 01:16:00it became clear what we were working for and how we were pricing it, other people would come in and say, no no, this is actually where you can get it cheaper with the government contractors here, or as a government we can get this kind of discount. So we're working on those types of problems as well but, again, emphasis on short turnaround.

The one thing that's quite different from say the more academic or research is that none of this was really written up in any papers. We wrote very few papers. We did publish a few. Most of this in terms of communicate, we developed--in fact it developed from the 2009 H1N1 influenza pandemic, a system where we write it out as a memo in a very structured way. In fact, the first elements in the memo is here's the question we are about to answer or perhaps a comment or two, not much writing, and here's your number one result. How we got there is of interest but the first thing leadership have--they haven't got time, they're 01:17:00busy, they've probably got a headache, who knows? A lot of pressure, they've got a phone call in five minutes, can I give them a number that will help with that as they discuss some aspect? Yes. Look at that table or graph. Is there two or three things about what we calculated--important in the assumptions? This is the number one assumption we made. We don't know how accurate. Just remember that as you're looking at that number. Then there's a long appendix where we explain everything in detail because I also know that somehow the top two pages can somehow get detached and people don't realize how it's all done. But if this goes into somebody, we don't know, and they're saying I don't know to trust it, there's at least the backup if they're interested. In fact, some of them ran as much as sixty pages but most of it was appendix and most of it didn't get read. But if somebody had a question, they'd say, I've got your memo, it says version nine on it. Okay, what question do you have? I'd say, I don't remember this, okay, on page sixty, okay, here's what we assumed, now I remember.

So it helped us remember. It helped us provide a point of conversation and clarification when asked for but, again, the first thing anybody saw was what's the key question, what's the key result, what's the key assumption going in and then lay it down with all the details. So a lot of what we did though meant that the public never saw it. Not we had choice but because this pace at which we were working didn't allow us to write--carefully sit back and write that erudite paper suitable for publication in the top-rated scientific journal. We did do some of that stuff but not a lot.

Q: When I think of the value of doing so, I think of the possibility for people learning from it and using it in the future, but like the Ebola response model specifically or any other thing that you worked on during this, what do you think the lasting impact can be of that work? Maybe even in your own work.

MELTZER: I think, again, you always learn about what are the ways that you can put together a model that will really help a decision-maker, and sometimes it comes down to sheer personalities. One decision-maker likes this way, another decision-maker likes that way, some decision-makers like more numbers and some like less. Some absolutely hate a given phrase and some say, I like that idea. So part of it is this communications part directly with--one, again, lessons that we learned not from Ebola but we applied it from previous outbreaks was we always should be down the hall ready to answer a call and we had many a midnight conversations with leadership about what did we mean about this number or can we possibly model? Yes, we can or no, there's no chance of us coming up with anything you're going to believe, here's three quick reasons why. This kind of conversation is the ability and willingness to do it is one of the most 01:20:00important things that you can do. In fact, a lot of what we wrote, we carefully labeled as pre-decisional. It wasn't the definitive decision, it was just part of the train of thought getting to the decision process. We never made a decision for anybody. When we had the time, and when it was really important like the Ebola response, yes, we did take the time out to write something and publish it because this really did--the public needed to know we're making decisions based on these estimates, here's how it was calculated and yes you can see as much as we can show you. We want to show you everything. We're going to have a go at showing you everything. Nothing to hide. But you can't do that at every point because that's just too much time and effort. You get distracted and writing for science journals is not the same as writing something or explaining something to decision-makers. We did do writing again, for the risk of number of Ebola cases imported into the US which obviously is very interesting and of 01:21:00major importance to people in the country and Congress and the media. We did publish a short note there. We also published a paper looking at the risk of spread of Ebola beyond the three countries in West Africa, not just on a plane to the US but to countries surrounding it and to other countries in Africa. We published and, in fact, we then did updates as well for that. So it's not that we never published but communications, we do have to make choices about where we're going to focus most of our efforts in communications and you have to be draconian on it. You cannot please everybody all the time. All you'll do is you'll actually do a poor job. You'll overtax the resources that you have and after a while, quite frankly, you'll lose control. It's better to know what you know and to understand what you haven't worked on and say, okay, that's not us, we haven't done that, we can't do it, or we don't have time to do it, and prioritize. Emergency response is all about prioritization and it does include 01:22:00decisions on what you're going to model and not model.

[break]

Q: Again, just pulling back to look at the timeline. Let's see, September 26th is when the MMWR comes out with the Ebola Response model and then--

MELTZER: Which was a busy day for us when it came out because the press knew about it. We spent a lot of time talking to the press and, of course, a lot of it is you've got to be kidding me, these numbers are not--and we said, look--the number that caught their eye was the now-infamous 1.4 million. So they said, you've got to be kidding and, of course, they get people's comments about CDC have drunk at the well of craziness, Meltzer is not to be trusted. All those type of things. We said, yeah, but there's also the part says what happens if you have a response, and in fact it turned out, which we couldn't know, we were forecasting five and half, six months ahead which is one of the longest forecasts out there at the time, and it turned out when you took our estimates 01:23:00of what would happen with the response, we were within 23% of what actually happened. Of course we couldn't know that, but we said we're pretty sure that if you do these interventions properly to the degree that we put in the model, you will stop it. In fact, I was asked point blank by leadership, do we think, in Liberia particularly, it will be over by Christmas? I said, no, I don't think you can bet on that but I think if you move fast and you do it correctly as best you can, and hit that target of 70% of the cases either in Ebola treatment units or at home with proper isolation and safe burial we needed, if you get to the point about 70% of all cases are in one of those two categories, by Christmas you will have not finished but really under control, which is what happened. And we actually have done some post-Ebola epidemic analysis to suggest that we came pretty close to the 70%.

So the accuracy is something you determine afterwards. You can't guarantee accuracy upfront. You'll be foolish to do so. But you do get a sense as you publish and people are asking, they're focused on what seems to be the spectacular, they don't always focus on the details. Leadership didn't. In fact, in that paper there was one other graph that was really important which showed the cost of delay. For every month in delay you'd have three, three and a half times increase in number of cases at peak. The important thing about that graph was when we originally did the first draft of the paper, that graph was not in there. That was done at the express request of CDC leadership because they wanted that piece of information to help explain to other people, Congress, the media, why we needed to go in big, but also why we needed to go in fast which is what that graph showed. So there you have a perfect example of leadership saying, okay, you're giving me something useful but there's something still missing. Can you do this? Sure, we can do this. Originally, the request is can 01:25:00you talk about every day in delay and we said no. Given the quality of data we can't break this epidemic down into per day, it's just too difficult, but we can give you in terms of every month of delay and they said, okay, that's enough. And it was used, I know for a fact, in intimate discussions at the very highest level regarding the need to move fast and it did help influence. So as much as people focused on the 1.4 million, what was more important, said, yeah, okay you scared us enough, what can we do about it? And the most important part of the model was yes, the publication September 26th, 2014 showed, yes, this isn't just fear mongering, here's the problem but here is the solution. Which was as I say difficult explaining to people because it's not a simple one-sentence explanation.

Q: And I have heard that, as you said, the model of what can happen if we 01:26:00actually intervene and what can happen if we intervene quickly, was used maybe even with like President Obama?

MELTZER: Absolutely. I like to believe--I don't necessarily know it's true but he came down to CDC in October to announce the whole US effort. Was it October or September?

Q: I think it might have been late September.

MELTZER: Late September, that's right.

Q: The military coming in.

MELTZER: Yeah, right, announced the big push and part of that reason, the rationale for announcing it, why they made the decision I've been told was because the numbers convinced. It wasn't the only story and it was also understood by leadership that, as I said, all models are wrong, some are useful. So the model showed we didn't know how accurate until after the event would be and some of that model, obviously, we will never know how accurate. We don't 01:27:00know what would've happened if interventions had stopped, if nothing had changed, because fortunately that didn't happen. But we do know that as far as I've been told that those numbers made their way up all the way to the highest level of decision making and helped convince leadership that we needed to go in big and we needed to go in fast. That's about as much as you can expect from simple models. So I was very proud of being part of that, proud of my team that we were able to help leadership the most when they needed it the most.

Q: Absolutely. I know I've kept you here sitting for quite a long time now. So after this publication, does the question pretty quickly become that of assessing individual risk?

MELTZER: Not so much. We would like--you know--goals can go in and identify 01:28:00somebody but there wasn't the data to go down, you said individual risk if you mean can we identify somebody is a risk. We know some of the risk factors, but a lot of that is epidemiological data which I think we're still sorting it out. If I were to redo the model now knowing what I know now, one element I'd like to actually incorporate or at least consider, assuming we have sufficient data, is that I believe now we're seeing from the data analysis, but only now, so a year and a half after we built the model is that there's a different level of risk of transmission within a household versus between households. Makes sense. Obvious in a sense, but we didn't have any data and we just said we have seen nothing. We did know before even when I built my model in previous outbreaks that burials is a high risk. It still is and was. One of the most important thing we did do, as I mentioned, was institute methodologies that allowed and even forced safe 01:29:00burials. Very important in breaking the chain of transmission. But we didn't know enough at the beginning about the different levels of risk of somebody to you--if somebody in your household is ill versus my neighbor is ill. I think there is a different level of risk and the reason why that's important is that when you get to a very big outbreak, you might want to say I can't be everywhere at once, I can't do everything, I'm going to concentrate on the number one risk factor which is within a household and focus most of my attempts of contact tracing, of getting hold of people and said you've been near somebody who's ill, we want to monitor you, monitor your temperature, monitor to make sure if you get sick, take you to an ETU [Ebola treatment unit] or something like that as soon as possible, focus those efforts on people within the household or very closely associated. I've now built that into the model or at least considered it, whereas before I considered it but I said there's nothing around, there's no 01:30:00data, I could make the model but I couldn't populate it with any believable data, and it would just illustrate to leadership what they know already is that we don't know anything about it. So obviously it might be important but we don't know how important. Now, we've probably got some data to suggest the degree of importance, so we could include them. But individual risk, that's about as low to individual risk that we can get to.

Q: Right, sure. I think I might have actually misspoken. You talked about how at the end of 2015 your--excuse me--2014 a lot of your effort began to focus on residual? Is that what I mean? Residual risk--

MELTZER: Residual risk. No, no, it's fine, actually they're both valid topics. Especially past the holiday period of 2014 and Liberia was definitely on the 01:31:00down slope and Sierra Leone as we got into late December beginning to bend down as well. We'd past the peak. In other words, skiing downhill. Particularly when you get past February and March particularly in Liberia, how quickly will it end? Well, the trouble is you keep getting these pop-up cases. What's the course of them, can we determine where they come from? And there we spent a lot of time determining on how much of those cases, what was the risk particularly of sexually transmitted cases? We knew that such cases likely occurred both from previous outbreaks but also back in September of 2014 when we were fitting the model, sometimes even the cumulative cases, there'd be spikes in the case numbers and we just couldn't fit the model to take into account of the spikes nicely, and the reason would be that these cases were occurring and previous 01:32:00history, the number of cases before, didn't explain these spikes. What it meant really was that there were cases occurring that had become infected--clinical cases of Ebola, but the cases that caused them were not being recorded. So you had these pop-up cases. Of course, you don't just get Ebola from nothing and they'd been infected by people who hadn't been recorded, part of this underreporting. Once we got four hundred cases a week, these cases just disappeared in the noise, it didn't make a difference. But when you're back going down and decreasing, all of a sudden you're getting more of these pop-up cases. Some of it, even after months of Ebola there were people who hid cases from the authority, who did not comply with safe burial, which tells you a lot about the difficulties in public health. Cultural responses and trust of government and building that trust and being effective. So even if somebody says 01:33:00I'm willing to listen to what the government and acquiesce to what the government wants to do, is the government able to get a burial team quickly to the house, for example, and do what--or are they going to wait days, which people would say that's unacceptable, we're going to do what we're going to do. Those type of things. So there was clearly some of that, but some of them--clearly wasn't a hidden case, they--where did they come from? Couldn't find any real explanation. Of course, as the number of cases declined, so you had more time and ability, your field of vision literally became clearer and you began able to really investigate the chain of transmission of every case or nearly every case. And sometimes there would be--we have no idea how this person got ill. They can't tell us. Well, began to suspect and do enough detective work like Dr. House used to be on the TV series right, find out that perhaps this person did have a case of Ebola because of sexual transmission. So then the 01:34:00worry was how many cases are there? Are we going to have a flare-up of disease, epidemic, because of these sexually transmitted cases or are they fairly rare? What's the degree of risk? So we did a lot of work on that as well.

Q: Okay well, do you think there's anything left to say? Anything that you'd like on the historical record about your work contributing to the response?

MELTZER: I think just sort of going over, again, it was exhausting. I think modeling proved its value. I think also it demonstrated the way we did it. There's some very strict criteria or operational conditions under which it does prove you have to work close to the decision-makers. This is not a matter about scientific publications, it is a matter about producing memos or the equivalent 01:35:00of that for pre-decisional--this isn't about being nice and neat. This is about telling the decision-makers what you know, how you knew it. This is about making the data accessible to decision-makers and the tool, explaining yourself. All of these things we sort of knew, this proof of the pudding, and I believe we have a system and a methodology that is worth duplicating and replicating in future outbreaks. There's always tweaks. Every outbreak has got a different flavor but the concept still holds, I believe is valid, and it should hold for many decades to come until we automatically collect all the data and we don't need modeling. But so long as we have uncertainties, you'll need the modeling. That's all.

[break]

MELTZER: One of the things that some of my team thought, we should've gone out to Liberia when we realized the data gaps and go and collect data. I didn't 01:36:00allow it. I said because I know what's going to happen when you get off the plane. Day-to-day supervision transfers to the CDC in-country chief and they're going to say, well, we know why you're here but guess what you're really doing? And I can't stop that. That's the way it is, that's realistic. And they're going to take one look at your skill set and say, you go in that little room and enter data. I want to collect more data. Oh, no, that's very difficult. Some of my junior colleagues found that very difficult. Subsequent we sent them out on a project and they found some data but then getting permission and all that, they suddenly realized I wasn't joking. The data might indeed be out there but it doesn't necessarily mean you're going to get your mitts on it, and even if you do, it doesn't mean to say you're going to be allowed to use it. So it seemed like an easy natural thing but, of course, I keep saying this can't be right and 01:37:00I said that's what we've got to do. There's a better way to do it. I said, well, if you show me--oh, if I go over there, I'll be able to do this. I said--frustrations build up clearly and it's not easy to quickly find ways around it, here's a solution. So you do have to be very disciplined in the approach. That also means you're not out there to make friends.

Q: No doubt. And you have to have familiarity about how stuff actually works.

MELTZER: A lot of that when you say it works is personality. [unclear] One of the biggest challenges we had was that the country directors did a lot of turnover. So we had [unclear]--I wanted to send somebody out and do modeling, if you want, they said, oh--most of them didn't really have familiarity with modeling to know how to use it but there were a couple and they said, yeah, let me get back to you, I'm just settling in. Get back to them next week, I said, I'm willing to send somebody out, they can be out there in a couple of days, 01:38:00because one thing about Ebola is to hell with travel, this, that and the other--

[break]

Q: And we're back to addressing other questions that came up--some of the most interesting stuff always happens after you turn off the recorder. [laughter] We were talking about real, practical challenges of the [unclear] and the response.

MELTZER: So indeed, some of it is just the pressure. Of course, you're working in a team. It is a pressure cooker so you've always got to worry about things like team morale. Some people get disappointed or upset about things like the quality of data or we ask too much, or this is just too difficult conditions, how do you deal with that? So, there's always at the end of the day all your people are persons, so as much as you just want to treat them as pawns, if you want to get the best results out of them, and we did, you've got to realize that this is stressful for them. Working within the model, of course, a lot of the teams saw the problems with all the data and being smart people that they are 01:39:00they said, we can fix this, we can fix that. I said, the problem there is we're not in charge of this. You have to learn the discipline that we are a cog in the machine. It might be an important cog but we're not driving the machine and so we can suggest things. We have to accept the fact that in the middle of a response with lots of noise, people interested, a lot of our suggestions just aren't going to be accepted partly because they might not see the value of it and partly because they're just too busy to absorb it. So somebody might suggest to me, and did suggest, should we go over to the West African countries and collect our own data? Well, sounds like a good idea except practically I knew it wouldn't work, but when I said, no, that's not what we're going to do, that builds up some frustrations because it feels like an avenue of solutions in improving the quality of data has been blocked. The other challenge that I think is very difficult--

Q: Let me ask you this one quick thing. How did you know that it would not work?

MELTZER: Well, simply because the process at which CDC was operating, particularly in September/October/November/December is that we did have, and still do have, CDC teams in country, but because of a number of regulations well beyond our control we tried mightily to change them and they were changed after a while but initially they weren't, was that people couldn't stay out in country for more than thirty days. So we might have a country director who actually knew about modeling and liked the idea and I'd talk to them and say would you like some help with modeling, can somebody come out and help you? They'd say that's an interesting idea but I'm going to deal with other things, I'm just learning the ropes. Called back a week later and said, how about that? Said, I'll think about it. Finally got to talk to them about modeling, they'd say, I like this idea but I'm leaving next week. So we're dealing always with data collection 01:41:00that is often a long term solution but we've got high rapid staff turnover so continuity was always an issue and that means when you have these type of problems, continuity, you go back to basics and you make sure that the very essentials you're going to keep doing it. It's not a good idea in the middle of such a stressful hectic chaotic situation to chop and change. If you make a change, you better be very sure that it's going to improve the situation, not just interesting. You must absolutely convince people it's worthwhile doing.

The other issue on that, obviously, is that even back in Atlanta people running the taskforce, their composition would change because people would volunteer for a detail for thirty days or sixty days, change over. Ensuring continuity--our team actually probably had the most continuity of all. I stayed as task leader and unit leader for the entire outbreak. In fact, even today I'm technically 01:42:00still on task so I've been doing this for over eighteen months. But other people come and go because there are other parts of the agency; the rest of the agency doesn't stop because of Ebola. They still have to continue the other tasks so people swap in and out and so you're constantly having to train people up, this is what we do as modeling, this is what we need from you or this is what we can give you.

And of course, then there's personalities and some people believe certain things should be done this way or this is better than that. Getting in sync with them and understanding what they need or what we can do for them is always a big challenge, and I don't think there's an easy solution to this. This is just bottom line is you've got to learn to be flexible, but it is stressful. I would say the leadership of the response learned over time is that you can't expect people to just carry on over and over again day after day without some breaks or 01:43:00something that relieves the pressure. One of the things I was allowed to do, for example, is we stopped our day job. We did respond occasionally but everything was, well, it'll have to wait until after--in fact, we're paying for it now, catching up, restarting the non-Ebola engine as it were, is part of it, but there is cost and since our whole unit was involved, every single project that we were working on just about ground to a halt. We answered a few phone calls and emails about it, but didn't do any major work with any other project simply because, yeah, it's not fair to the team to expect them to work ten hours on Ebola and then go home and work from home on their non-Ebola job. That's just too much. Just trying to get some time off, and one of the things my team really craved after a while was they wanted to go back to their cubicles because it was 01:44:00nice and quiet compared to our noisy team room--and I said, I know, but we're effective in this room, people came and found us, we could bring them up to us, I know where you all are if I get a phone call and can I have an estimate of this in the next couple of hours and I can assign it to the people around the team like that, I don't have to find you. They did appreciate it. Didn't make them any less unhappy when thinking about working under the stressful conditions. So some of those challenges that don't go away you just have to learn how to deal with them.

Q: Is there anything you would've dealt with differently?

MELTZER: It's hard to think of something. I'm sure if you ask my team members they might come up with some ideas like, yes, get rid of Meltzer as leader is the first one. No, I think nothing's perfect so you can always think of ways of perfection but you've also got to say how much would that realistically have worked? I think we did work very well. I think one of the reasons it worked well 01:45:00was that we were quite disciplined in terms of what we did, how we did it, and we didn't go chasing every single idea. We did say no very often. We tried to say yes to every important question. We always asked how is this going to be used? So people often said, oh, this would be interesting. We said, no, we're not doing that. If a leader came along and said I need this estimate because we're making this decision, that's obviously important. So from that point of view it was good. Always you can say could relations between our taskforce and other taskforces could've been better? Sure. But it's hard under the stressful conditions to say you can guarantee those things would've improved. I think the one thing that I would've pushed even harder that we pushed hard was the data collection efforts in-country in West Africa. There was a long complicated form. We did ask and we tried absolutely to have it shortened and condensed because we thought that would be more efficient. I think, the next time around I'll really 01:46:00stick to my guns and absolutely insist upon it so we can't function with anything other than something more focused with specialized studies to collect the additional data that are needed. But there was an attempt essentially, especially in the early days, to try to collect every bit of data from every patient. It really clogged up the machine and made life difficult not just for us but for everybody. In fact, it would've been better to pass it down to the essential index card information for every patient with specialized studies for others. But it was a hard sell. We tried really hard. We didn't make the sell. Would like to think were to do it again or in the future that somehow I'm going to focus on one of the most important things we can do as a modeling unit is really influence what data are collected and how they're collected. It does mean in fact we should be there even sooner than when we're called in right up front and saying, okay, for the first ten patients you can collect whatever you want 01:47:00but after that you're going to be much more parsimonious and more efficient in what you collect. Not necessarily an easy thing to sell because we did try it, as I said, in November. Didn't make the sell. So could try again, can't guarantee I'll make the sell next time either. Anything else, Sam, at this point? Feel free, by the way, in the future to come back with additional questions.

Q: And I'm sure I'll have them. So I'll release you. [laughs] Thank you very much.

MELTZER: Thank you for your time and patience.

END