1002 UNITED STATES DISTRICT COURT FOR THE DISTRICT OF COLUMBIA ELOUISE PEPION COBELL, et . al. . . Plaintiffs, . Civil Action 96-1285 . v. . . Washington, D.C. DIRK KEMPTHORNE, Secretary . Tuesday, June 17, 2008 of the Interior, et al. . 1:33 p.m. . Defendants. . . . . . . . . . . . . . . . TRANSCRIPT OF TRIAL - AFTERNOON SESSION BEFORE THE HONORABLE JAMES ROBERTSON UNITED STATES DISTRICT JUDGE APPEARANCES: For the Plaintiffs: DENNIS GINGOLD, ESQ. Law Offices of Dennis Gingold 607 14th Street, NW Ninth Floor Washington, D.C. 20005 202-824-1448 ELLIOTT H. LEVITAS, ESQ. WILLIAM E. DORRIS, ESQ. Kilpatrick Stockton, LLP 1100 Peachtree Street Suite 2800 Atlanta, Georgia 30309-4530 404-815-6450 KEITH HARPER, ESQ. JUSTIN GUILDER, ESQ. Kilpatrick Stockton, LLP 607 14th Street, NW Suite 900 Washington, D.C. 20005 202-585-0053 APPEARANCES con't. on next page. Jacqueline M. Sullivan, RPR Official Court Reporter 1003 APPEARANCES, con't. DAVID C. SMITH, ESQ. DANIEL R. TAYLOR, JR., ESQ. Kilpatrick Stockton, LLP 1001 West Fourth Street Winston-Salem, North Carolina 27101 336-607-7392 For the Defendants: ROBERT E. KIRSCHMAN, JR., ESQ. JOHN WARSHAWSKY, ESQ. J. CHRISTOPHER KOHN, ESQ. U.S. Department of Justice 1100 L Street, NW Washington, D.C. 20005 202-307-0010 JOHN STEMPLEWICZ, ESQ. Senior Trial Attorney U.S. Department of Justice Commercial Litigation Branch Civil Division Ben Franklin Station P.O. Box 975 Washington, D.C. 20044 202-307-1104 Court Reporter: JACQUELINE M. SULLIVAN, RPR Official Court Reporter U.S. Courthouse, Room 6720 333 Constitution Avenue, NW Washington, D.C. 20001 202-354-3187 Proceedings reported by machine shorthand, transcript produced by computer-aided transcription. Jacqueline M. Sullivan, RPR Official Court Reporter 1004 1 P R O C E E D I N G S 2 COURTROOM DEPUTY: Please come to order and remain 3 seated. 4 THE COURT: All right, Mr. Dorris. 5 MR. DORRIS: Thank you, your Honor. 6 FRITZ SCHEUREN, Ph.D., WITNESS FOR THE GOVERNMENT, 7 PREVIOUSLY SWORN 8 CROSS-EXAMINATION 9 BY MR. DORRIS (continuing): 10 Q. Good afternoon, Dr. Scheuren. 11 A. Good afternoon to you. 12 Q. Let me move to a little different subject than where we 13 were before the lunch break. You testified some about changes 14 that were made in some of the information that was given to us 15 from -- some information that was provided on May 30th to us and 16 then information that was provided to us on June 4th. Do you 17 recall testifying about that during your direct? 18 A. I do. 19 Q. I'm going to ask that we bring up on the screen Plaintiffs' 20 Exhibit 117. Maybe I've got the wrong number. 21 Can I have just a minute, your Honor? 22 (Pause in the proceedings). 23 MR. DORRIS: I'm sorry, your Honor. I had written 24 down the wrong exhibit number. Oh, here it is. Do you show 25 what exhibit number that is? Okay. Jacqueline M. Sullivan, RPR Official Court Reporter 1005 1 BY MR. DORRIS: 2 Q. I believe it has been marked as 117. And in looking at 3 this I want you to look at the bottom two that are referenced at 4 AR-171R1, the May 30th, and then AR-171R2, which is the June 4. 5 Do you see those? 6 A. Yes. 7 Q. Okay. And if you look over, and this is just revenues, 8 okay, this doesn't address disbursements or other information. 9 Do you see that between the information we were provided on May 10 30th showed total revenues in the pre-1972 time frame of almost 11 $3.4 billion? 12 A. I do. 13 Q. And then in as of June 4, 2008, they'd fallen to about $2.8 14 billion. Do you see that? 15 A. I do. 16 Q. Okay. Now, I don't believe that there were any changes in 17 what we were provided between those two between 1972 and the end 18 of the chart, which I think is 2007. Do you know whether there 19 were any changes in that later time frame? 20 A. To my knowledge, no. 21 Q. Okay. So all of the changes that occurred between these 22 two runs or whatever occurred -- made changes in the pre-1972 23 time frame, fair? 24 A. Yes. 25 Q. Now, you testified, and I tried to follow but I couldn't Jacqueline M. Sullivan, RPR Official Court Reporter 1006 1 exactly, why there were further changes made between May 30 and 2 June 4, 2008. Did I understand you to say that the process you 3 were going through simply wasn't complete on May 30, 2008? 4 A. That is correct. 5 Q. And did you get it completed by June 4th? 6 A. I believe it is. 7 Q. Now, what changes, if you know, were made between those two 8 analyses that would cause about $600,000,000 in revenues to go 9 down during that period, the pre-'72 period? 10 A. We did two things. I already said some of this to you. We 11 looked at the imputation model and we improved it. We were not 12 happy with the -- well, with how good it was working for us, and 13 we improved the model. And then we also looked at the reported 14 data, okay, which we had not factored into the uncertainty in 15 the unreported data at that point. 16 Q. Okay. So you looked at it, the model, because you weren't 17 happy with it, correct? 18 A. That's correct. 19 Q. Why weren't you happy with it? 20 A. The disbursement -- it was getting negative values for the 21 balances, and this is over the whole life of the trust, that 22 couldn't be right, so we tried to find out what it was, and I've 23 already mentioned to you that one of the -- it doesn't sound 24 like it's a big deal, but one of the things that hit us was we 25 had used the value in 1887 as if the trust started in 1887, Jacqueline M. Sullivan, RPR Official Court Reporter 1007 1 which in fact in some senses it didn't. 2 Q. Oh, okay. So that was one thing you did talk about, how? 3 A. This morning. 4 Q. And the model that was run that generated the May 30th 5 information we were given, are you saying that that was 6 presumably a zero balance as of 1887? 7 A. That is my recollection. 8 Q. Okay. You've never produced the analysis for May 30 to us, 9 have you? 10 A. No, I have not. I think I also said this morning that we 11 used only a small number of iterations, only a thousand, in the 12 May 30 data. Like ten thousand. 13 Q. So between May 30 and June 4 we've identified a couple 14 things, and keep me straight. We've identified that you've now 15 put a balance in for 1887 and you ran ten times the number of 16 imputations, correct? 17 A. I didn't put a balance in. I did -- I didn't force a 18 balance of zero, okay, in 1887, which we had done earlier. 19 Q. Okay. You didn't put a balance in, but you let this model 20 that you created generate a value for that missing information? 21 A. Correct. 22 Q. And it generated a value of $5.8 million balance in 1887, 23 correct? 24 A. Yes. 25 Q. Now -- Jacqueline M. Sullivan, RPR Official Court Reporter 1008 1 A. That needs to be footnoted, though. 2 Q. That needs to be footnoted? 3 A. Yes, because of the way we did this. 4 Q. Can I ask this, it's not footnoted on anywhere on what 5 you've done, is it? 6 A. No. 7 Q. Okay. You said it needed to be footnoted. Would you say 8 what you mean? 9 A. What I need to do is explain to you that we actually had to 10 make a starting value, okay, in this area, and grew out of the 11 model that we chose, which we had to go back seven years, as you 12 remember this from this morning, and were not showing all those 13 seven years. We're just showing first of the years, 1887, so 14 that number is coming from, you know, coming from a process you 15 haven't fully seen. 16 Q. Okay. Now, so there was some analysis, further analysis 17 between May 30 and June 4 for a time period between 1880 and 18 1887, correct? 19 A. Yes. 20 Q. Where you were finding that or establishing in some fashion 21 that there was money coming into the government for individual 22 Indian allotted lands, correct? 23 A. I don't know how to answer that. We were not really doing 24 that. We were trying to fix this model so it worked for us. If 25 you look at these numbers here you will see that the collection Jacqueline M. Sullivan, RPR Official Court Reporter 1009 1 numbers for 1887 were 2.2 million and disbursement 2.1 million 2 and that meant there was a balance increase for that year of 2.1 3 million, so where does the other money come from? It obviously 4 comes from the past, okay. It has to come from the past, and 5 this issue is one that we're -- we don't have -- we actually -- 6 it comes out of the process that you haven't seen. 7 Q. Okay. And this process I haven't seen would indicate to me 8 that on average over a course of six or seven years there is 9 approximately $5.8 million balance that is generated through 10 that process, correct? 11 A. That is what is happening, yes. 12 Q. Okay. Am I correct when I look at the top line of 13 Defendants' Exhibit 462, that collections of 2.2 million less 14 disbursements of 2.1 million ends up with a balance at the end 15 of fiscal year 1887 of $5.8 million, am I reading the chart 16 correctly? 17 A. You are correct. 18 Q. So the beginning balance at the end of fiscal year 1886 19 under this would be 5.7 million, correct? 20 A. If it's footed, yes. 21 Q. Now, for those seven years then the average amount would be 22 approximately eight hundred to nine hundred thousand dollars in 23 a buildup of the balance, correct? 24 A. I think I need to give you more information than this, but, 25 yes, that is a logical conclusion, yes. Jacqueline M. Sullivan, RPR Official Court Reporter 1010 1 Q. And then after that there isn't another year where this 2 model that you have constructed would show that much build up in 3 the balance until which year? 4 A. Well, in looking at the data it looks like 1903. 5 Perhaps -- I'm not sure what you mean. 5.8 to 10, what is that? 6 Q. Well, it looks like to me there's a $400,000 difference 7 between collections and disbursements in 1903. 8 A. Yes, but you're not looking at the balances. The 9 difference between the two balances is $3 million. 10 Q. Oh, okay. So you're saying, well, look at the balances, 11 don't look at the collections and disbursements? 12 A. Well, I won't say don't. Don't just look at the 13 collections and disbursements. We do not -- this system does 14 not foot until 1996, and the balance information is in the 15 system but it's not -- is tied -- perhaps it's timing or 16 whatever it is, but it's not tied closely. 17 Q. You said it didn't foot until 1996? 18 A. That's correct. 19 Q. Did I understand you correctly? 20 A. That's what these data show. 21 Q. That's what this model shows, correct? 22 A. Well, there is two ways to answer that. One of them is in 23 those years we have data, collections, disbursements, and 24 balances. The only years which we have data in foots begin in 25 1996. Jacqueline M. Sullivan, RPR Official Court Reporter 1011 1 Q. Okay. Now, is that something that you set the model up to 2 do, to make sure it footed as of 1996? 3 A. No. It does. That's real data. 4 Q. Okay. Then let me ask this: The data for 1996 is real 5 data, is what you're saying? 6 A. Yes, sir. 7 Q. And is that different than the other data that you show 8 here as reported values? Are you treating that different in 9 some fashion? 10 A. You're right in that some of the data, the data that's in 11 the boxes, okay, was reported data, but has been model adjusted. 12 The data that was not in the -- does not have a box around it 13 and is not color-coded is reported data. 14 Q. Okay. So when I look under the columns Collections and 15 Disbursements on Defendants' Exhibit 462, everything that was a 16 reported value earlier now shows as being a model adjusted value 17 all the way up until 1996, correct? 18 A. When we're talking about collections and disbursements, 19 what you said is true. The balances were not model adjusted. 20 They were as we left them after imputation. 21 Q. You didn't try to adjust those in the same way that you 22 tried to adjust collections and disbursements, correct? 23 A. That's correct. They were there to help us model 24 disbursements and collections. 25 Q. You said something during your direct examination, and I'm Jacqueline M. Sullivan, RPR Official Court Reporter 1012 1 talking from my notes, not from the transcript, but it was 2 something about that the model was so unstable that you treated 3 1996 and beyond as actual data, at least that's the gist of what 4 I heard you say. 5 A. That is not what I said. 6 Q. Okay. Is that true? 7 A. Let me separate the truths that you're saying into pieces 8 and then see what the causal links are, if there are causal 9 links. May I start? 10 Q. Please. 11 A. The first thing to remember is that the 1996 data, which 12 was very useful, has been audited it. All the data since 1996 13 has been audited and all of it is footed. It behaves in the 14 proper way, is an accounting system. The earlier data does not 15 behave in that way, which made us suspicious of it. We took the 16 1996 data as useful in understanding the earlier data, but we 17 didn't change it. 18 Q. Okay. Did you say, did you talk at all during your direct 19 examination about any instability in the model as it relates to 20 these years 1996 and beyond? 21 A. No, not about that year. 22 Q. Now, so you are taking from 1996 through 2007 because there 23 are audited financial statements, you're accepting those as 24 being correct? 25 A. I'm accepting those, yes. The word "correct" is your word. Jacqueline M. Sullivan, RPR Official Court Reporter 1013 1 Q. Okay. When you say you're accepting them, is there still a 2 part of your analysis that assumes any uncertainty with respect 3 to the data from 1996 forward? 4 A. No. 5 Q. Okay. So you are not viewing that as being uncertain in 6 the least? 7 A. It is not part of the modeling process, correct. 8 Q. I did not understand what you said. 9 A. It is not -- our focus of the modeling. 10 Q. Can you just repeat your answer? 11 A. I can't do that exactly but I'll say it differently maybe 12 and that will help you. 13 Q. Okay. 14 A. And of course you'll have the transcript too to compare 15 what I said the first time to what I said now. 16 We know the data was audited in 1996. It foots, it 17 makes sense in terms of the earlier data, okay, for use earlier 18 and collections and disbursements. And we took -- we took that 19 data as a given and not a need of any further adjustment, either 20 an imputation adjustment, because there was nothing missing, or 21 model adjustment because of the possibility that there was some 22 errors still in that data. That is not to say that there 23 couldn't be errors in that data, we do not know that, but we 24 were so behaved so well that we did not do any modelling for 25 that. Jacqueline M. Sullivan, RPR Official Court Reporter 1014 1 Q. Okay. I got off on this as I was trying to find out what 2 went on in these two different analyses for the pre-1972 period, 3 and let's go back to what I'm calling Plaintiffs' Exhibit 117. 4 Now, you told me that part of the reason these changed had to do 5 with the -- that you included 1980 through 1986 in your 6 analysis, correct, is one reason that the numbers changed? I 7 may have said 19. I should be saying 1880 to 1886. 8 A. That would help me, yes. 9 Q. I'm sorry I misspoke. Can you explain any of the other 10 reasons that the numbers changed between May 30 and June 4? 11 A. We of course are fitting a different model, okay, than we 12 were fitting earlier, all right. A model which we liked better 13 and we were comfortable with, and also I mentioned earlier that 14 we had -- we have, subsequent to the multiple imputation, we had 15 done the model adjustments for the reported values, and I'm 16 pretty sure that those were not done until after May 30th, 17 although we may have done them, but the version you got on May 18 30th did not have them. 19 Q. Okay. And when you talk about the model-adjusted numbers, 20 those are the ones on DX 462 that show up in the black boxes? 21 A. That's correct. 22 Q. Okay. And certainly the addition and subtraction of those 23 don't come anywhere, the addition of all the additional revenue 24 reflected here, are you saying that that more is shown in those 25 boxes than that that explains a $600 million difference? Jacqueline M. Sullivan, RPR Official Court Reporter 1015 1 A. Let me try to answer the question in a somewhat different 2 way. I am not claiming that the difference, numerical 3 difference between these numbers is due to A, B, C and D. What 4 I am telling you is what A, B, C and D, which is what we did, 5 occurred between these two numbers. We went away from this 6 model. We were in happy with it. And the fact that we're 7 continuing to talk about it is your prerogative, but I'm not 8 happy with this model. 9 Q. And can you list to me the primary reasons you were not 10 happy with the model of May -- that resulted in the May 30 11 numbers? 12 A. Well, I told you two of them and I'll mention them again. 13 And to order them in terms of a report I don't know if I can do 14 that right now. First thing is that we're getting some negative 15 values from the earlier value. 16 Q. Say it again. 17 A. Negative values. The balance over the whole life of the 18 trust was negative, and that didn't make sense. That did not 19 make sense. And the second reason was that we had in 1887 20 things, so as soon as we got rid of the 1887 problem that we 21 treated as a zero the model behaved in a way that made sense, 22 and then we went on to do the model-adjusted steps for the 23 reported values. I didn't realize that I was going to have to 24 be asked to talk about this. I'm happy to talk about it of 25 course and then maybe I can give you a better answer later, but Jacqueline M. Sullivan, RPR Official Court Reporter 1016 1 this is the answer I have right now. 2 Q. Okay. Well, let's do move on. Have you ever applied this 3 multiple imputation model to a situation such as this one? 4 A. The degree to which the situations are similar is in the 5 eye of the be holder. I've applied multiple imputation in lots 6 of situations. Exactly like this, never. 7 Q. Okay. Now, as I have read your article that was produced 8 today and other articles on multiple imputation, it originally 9 grew out of trying to impute values for missing data in 10 extremely large surveys, correct? 11 A. Correct. 12 Q. And those values needed to be imputed in those surveys so 13 that then-existing techniques for studying a completed data set 14 could be run on those sets, correct? 15 A. That is correct. Those techniques of course are still 16 being used. 17 Q. And that's the reason why in those surveys that you want to 18 impute or put some value in for the missing data? 19 A. The value of the imputation, the multiple imputation -- 20 important word -- was to get to uncertainty, but the analysis 21 had to be done, especially given this offer in those days, was 22 to make the data look like a complete data set because the 23 methodologies for those complete data sets already existed. 24 Q. Okay. You say that similarities or whatever are in the eye 25 of the beholder. Let me ask you, as the beholder, can you Jacqueline M. Sullivan, RPR Official Court Reporter 1017 1 describe for us the closest situation to this that you have ever 2 used multiple imputation for? 3 A. I wish I had known you were going to ask that question. I 4 would you have thought about a better answer, but most of my 5 applications have been in administrative records when I was at 6 the IRS, and in surveys. 7 Q. Okay. Would it be fair to say that this is the first time 8 that you've used multiple imputation in trying to develop 9 missing values for a trust or banking situation? 10 A. That's a bit broad, but it's certainly true that I've not 11 done this kind of analysis before. 12 Q. Now, when you look at -- you used a phrase during your 13 direct, a word that I guess is one of those words that 14 statisticians come up with. You said the "missingness" of 15 certain data. Do you remember using that word? 16 A. Yes, I did. I do use it, yes. 17 Q. Now, if you would bring up Defendants' Exhibit 461. Can 18 you describe to me what you mean by the missingness of data? 19 A. Well, there are multiple measures of that that you can 20 have. One of them is simply to look at the number of missing 21 cells relative to the total number of cells. Another one is to 22 look at the dollars in the imputed data and in the reported 23 data, how many of the dollars have been imputed. That's another 24 kind of degree of missingness, okay. 25 Q. Okay. Let me ask before we go through, you don't know what Jacqueline M. Sullivan, RPR Official Court Reporter 1018 1 the imputed information is when you've got Defendants' Exhibit 2 461, correct? 3 A. That's correct. 4 Q. And when you make an assessment, for example, of the first 5 page of Defendants' 461, how would you characterize the 6 missingness of the data on that chart? 7 A. Considerable. It's considerable. 8 Q. Is there a point where you have so much missing data that 9 multiple imputation is not a good device to use? 10 A. I'm sure there is. 11 Q. And we're at that point if this was the only page that we 12 had in Defendants' Exhibit 461? 13 A. I admit we are, yeah. 14 Q. Look at the next page. If I told you on the next page, Dr. 15 Scheuren, to assume that all of the values that are stated for 16 collections between the years 1927 through 1949 are incorrect 17 and you should disregard them, in other words, if all of that 18 was yellow, would you agree with me then that there was too much 19 missingness on that page to use for a multiple imputation 20 process? 21 A. 1927 to 1949, is that what you said? 22 Q. Yes, for the collections column. You have reported values 23 there and I've asked you to assume those are all missing and are 24 yellow. 25 A. Let me make two comments. First is to compare the first Jacqueline M. Sullivan, RPR Official Court Reporter 1019 1 page to the second page if you would let me, and then second is 2 to answer your question directly. 3 Q. Yes. Would you show -- 4 A. I don't need to see it again. There is considerably more 5 data here even if those, that second column, disbursement 6 column, were yellow, than there is on the first page. What is 7 weakened here greatly is the relationship between collections 8 and disbursements in that second -- well, it's the third column 9 on the page if the disbursement column were yellow, so my answer 10 here on this page was that it depends on what it is you are 11 trying to impute here. What's your final number? If you're 12 into, and we are, into a balance figure displacement, 13 collections minus disbursements, then I would be worried about 14 this probably if that were the case. 15 Q. Okay. To make clear, your answer when you referred to the 16 second column, you're referring to the column entitled 17 Collections, and when you refer to the third column you were 18 referring to the column entitled Disbursements, correct? 19 A. Yes. And your question was about the disbursement column, 20 I believe. 21 Q. No, it was about the collection column, that if you -- 22 A. Oh, I'm sorry. 23 Q. That if you did not -- let's make sure it's clear because I 24 think your answer is the same. But if on the second page of 25 Defendants Exhibit 461 they reported values in column two that Jacqueline M. Sullivan, RPR Official Court Reporter 1020 1 is labeled Collections were all missing between 1927 and 1949, 2 that was the premise of my question. Did you understand that? 3 A. Now I do. I misunderstood. I thought it was disbursement, 4 but I'll say it a different way as I think as you've already 5 anticipated my answer here. 6 If one or the other of these columns were all missing, 7 then what I said would be correct. 8 Q. And what you said would be correct is that then there would 9 not be sufficient information to use a multiple imputation model 10 to try to project a missing data, correct? 11 A. If I might interject a word, the word "strong model." I 12 can still model it, but I don't have enough feedback in the data 13 to have me believe the model. The data is not telling me that 14 the model can be relied on. You can always make strong 15 assumptions if you want, but I wouldn't have used that, no. 16 Q. So that when what you're saying is you could still model it 17 but you wouldn't find that to be reliable? 18 A. If you make too many assumptions you have to worry. 19 Q. Okay. Now, you've been in court on and off since the trial 20 started, correct? 21 A. Yes, I have. 22 Q. And are you aware of there being any questions or concerns 23 about the collection information that has been used in the 24 period from roughly 1922 or 3 all the way through 1949? 25 A. I'm afraid you're going to have to refresh my memory here. Jacqueline M. Sullivan, RPR Official Court Reporter 1021 1 I'm not sure what I was hearing. Some of the time I wasn't 2 paying as much attention as at other times. 3 Q. Let's pull up Defendants' Exhibit 372, and look at page 4 eight. 5 I've called out the wrong page. Go back -- well, hold 6 on just a second. Go back to page four, please. And then 7 let's -- 8 Your Honor, may I get my notebook? I'm sorry, I don't 9 have it at the podium. 10 THE COURT: Sure. 11 BY MR. DORRIS: 12 Q. Page ten, Defendants' Exhibit 372, page ten, and I'll tell 13 you what. If you blow up the top part so that Dr. Scheuren can 14 see that. And you see that this particular chart is referring 15 to the years 1934 through 1949. Are you with me? 16 A. Yes, I am. 17 Q. And then it gives a document reference to the far right 18 column, footnote two. Do you see that? 19 A. I do. 20 Q. Okay. And then if you would blow up footnote two. 21 Dr. Scheuren, we've read this many times. I would ask 22 you to read footnote two to yourself. 23 A. Yes, I've read it. 24 Q. Those indicate that the reports on which the collections 25 for those years are being based likely understate total Jacqueline M. Sullivan, RPR Official Court Reporter 1022 1 receipts. Do you see that? 2 A. I do, but I remind you that you're looking at tribal here, 3 not the total. 4 Q. Okay. 5 A. Pull back up to the top. You're looking at tribal. 6 Q. Okay. Let's go there. Go to 1934 and kind of blow up the 7 top year so we get it nice and large. 8 You say it's highlighted there. The receipts that are 9 being referred to there, sir, are receipts, those are receipts 10 that were used -- well, you tell me. What did you use in your 11 model on -- let's just look at this number. See the 10.6 12 million? 13 A. I see it. 14 Q. And that's for 1934. You're with me, the 10.6. 15 Let's pull up Defendants' Exhibit 461 and go to the 16 second page. 17 And do you see under 1934 you used as a starting, as a 18 reported value 10.6. Are you with me now? 19 A. I do see the coincidence, yes. 20 Q. That's not tribal, is it? 21 A. No, it's not. 22 Q. And do you see there where we saw a footnote there that 23 indicated that the information from which that was taken and 24 then a statement that Ms. Herman I believe testified that she 25 and Dr. Angel prepared together that says that all those revenue Jacqueline M. Sullivan, RPR Official Court Reporter 1023 1 figures are likely understated, were you aware of that before 2 right now? 3 A. I heard this business about understatement, of course. 4 This connection that you've just made between these two exhibits 5 I am hearing and seeing for the first time. 6 Q. Go back to page -- on Defendants' Exhibit 372 to the 7 first -- to page four. And blow up kind of, I need to go to the 8 other -- over to the Other Receipts column, the first seven or 9 eight lines. Do you see over there under the Other Receipts 10 column here on this front page where it is showing that you look 11 at schedule A-1 page one in order to get the numbers that are 12 there? 13 A. Apparently there is come kind of a footnote here that I'm 14 not seeing. 15 Q. Okay. And then do you -- 16 A. So my answer to your question is no, I do not see it. 17 Q. Would you point the arrow under Tribal IIM to the footnote 18 A-1-1? Do you see where the red arrow is pointing now? 19 A. I do. 20 Q. Do you see the footnote? 21 A. No, I haven't seen the footnote. Let me see the footnote. 22 Q. No, but you just see the reference to A-1 page one? 23 A. Yes, sir. 24 Q. And when we turned to A-1 page one in this exhibit, it is 25 page number eight. Are you with me? Jacqueline M. Sullivan, RPR Official Court Reporter 1024 1 A. I'm getting there. 2 Q. And then do you see for Tribal IIM, if you'll point the 3 arrow at it, the footnote, you see Tribal IIM and then you see 4 the footnote next to it that you look at page A-1.1 page one? 5 A. Um-hmm. 6 Q. And then if you go to page ten, and that's the page we were 7 just looking at with the note on it, do you see that? 8 A. Yes. 9 Q. And that has to do with A-1.1. So do you see how this 10 document fits together? 11 A. Yes, but you're moving a little fast for me. Remember my 12 vision is a little slower than that, but, yes, I see this. I 13 commend you for putting this together. Thank you. 14 Q. Let me ask it to you in a question this way since you're 15 testifying as an expert statistician. If I ask you to assume 16 that all of the collection information shown on Defendants' 17 Exhibit 461 as reported values from 1923 through 1949 are likely 18 understated, what modifications or adjustments, if any, would 19 you make to what you have done? 20 A. If that's the case then I would have wanted to have 21 adjusted them up to account for that before I did the rest of 22 the analysis. 23 Q. And that was not done, was it? 24 A. It was not. We did take into account uncertainty in the 25 data, as you heard me tell you, but we did not expect to model Jacqueline M. Sullivan, RPR Official Court Reporter 1025 1 any bias in the data. 2 Q. Okay. I don't understand the last phrase you did not 3 expect any modeling bias in the data. What does that mean? 4 A. I don't think I quite said that. 5 Q. Tell me what you said. 6 A. Suppose that every one of those values was off by $3 7 million, okay, every year. 8 Q. Off by what? 9 A. By $3 million, just hypothetically. You give me a 10 hypothetical I give you one back, okay? So off by $3 million 11 and I was able to learn that somehow. I mean, enormously expert 12 people in the room here at Angel and Michelle Herman, and if 13 they had come to me and said these data are off by $3 million I 14 would have adjusted them upwards by $3 million. I would have 15 told you that. 16 Q. And if they were not able to come up with a reasonable 17 estimate for you as to how understated those values were, what 18 would you do? 19 A. I would not have been able to have done any more than I 20 did. 21 Q. So you would have run them in the model just like you have, 22 even though they're -- 23 A. If I knew they were consistently understated I would have 24 developed an adjustment process that was asymmetric. The 25 process we used assumed errors in reporting in both directions. Jacqueline M. Sullivan, RPR Official Court Reporter 1026 1 I would have adjusted it only reporting errors in one direction. 2 I did not do that, though. 3 Q. All right. And those are adjustments that you would have 4 made before you ran the model, correct? 5 A. What we actually did, of course, is we took out the really 6 extreme outliers as we thought them outliers, and then knowing 7 they were reporting errors we went ahead and did the imputation 8 and then we went back in afterwards and adjusted the reporting 9 and imputed the data. And the reason we did that is because the 10 system is so sparse that we needed the imputed data to help us 11 to decide how to adjust the reported data and the imputed data 12 together. 13 Q. So that even with all of the reported values that you've 14 used such as they are, the data was so scarce that you actually 15 needed to run an imputation and impute values in order to then 16 make further changes to the model? 17 A. To the reported data, yes. 18 Q. To the reported data? 19 A. And the imputed data at the same time. 20 Q. Let's go back to Defendants' Exhibit 462. Do you see the 21 years 1909 through 1911 are now in black boxes as being model 22 adjusted values? Do you see that? 23 A. I of course do. 24 Q. That was some of the reported value you had before, 25 correct? Jacqueline M. Sullivan, RPR Official Court Reporter 1027 1 A. Correct. 2 Q. And explain to us why the model adjusted the values for 3 these reported amounts. 4 A. What we had done is we had fit a time series model to the 5 whole data set after we had done the imputation and identified a 6 degree to which the model did not fit, and then we took the 7 residuals from the model and we treated those as a way of 8 thinking about the remaining uncertainty in the reported values, 9 and then we used those residuals to increase the amount of our 10 measure of uncertainty. Remember that our focus is not really 11 on the model values at this level. Our focus is on the overall 12 uncertainty at the balance level. That was our goal. If we had 13 a goal of producing a best estimate for 1918, okay, collection 14 and disbursement, we would have done additional work localized 15 to that year. 16 Q. So if you were trying to come up with the best estimate for 17 19 -- let's just say 1912, what additional work would you have 18 done to try to come up with the best estimate for those years, 19 for collections and disbursements? 20 A. I don't know the answer to that right now. Do I have to 21 know the answer? I mean, you're asking hard questions. You 22 know, at least they have hard answers from my world. 23 Q. Well, an acceptable answer is to say "I don't know at this 24 time." 25 A. That's the answer. I don't know at this time. Jacqueline M. Sullivan, RPR Official Court Reporter 1028 1 Q. Okay. Well, let me ask this. When I compare the reported 2 values on Defense Exhibit 461 to the model adjusted values on 3 Defendants' 462, I actually find that the model adjusted the 4 collections for those three years up by about $100,000 or 5 $200,000 and the disbursements down. 6 Can you put the other one next to it? Thank you. I 7 don't know if you can then blow up those particular areas. 8 It's not a lot of numbers, but it's odd to me. Can 9 you see the screen at all? 10 A. I can see. What I hear -- 11 Q. It's coming. You see in 1909 -- 12 THE COURT: The story of this trial, you know, is 13 going to be the dualing laptops. These guys are both good. 14 MR. DORRIS: They are, and it makes it much more 15 efficient. 16 BY MR. DORRIS: 17 Q. And now, Dr. Scheuren, do you see, for example, what I said 18 in looking at DX 461 for the years 1909, 1910, and 1911 and 19 comparing that to those same years for collections in Defense 20 Exhibit 462, how each of those numbers has been increased by one 21 hundred or $200,000? 22 A. I do see that, correct, yes. 23 Q. And do you see similarly with the disbursements for those 24 three years from Defendants' Exhibit 461 to 462, the 25 disbursements were all decreased? Jacqueline M. Sullivan, RPR Official Court Reporter 1029 1 A. No. 2 Q. The disbursements were all increased by $100,000; is that 3 correct? 4 A. Thank you. Yes. 5 Q. All right. So why did the model make those kind of 6 adjustments, do you know? 7 A. Specifically why did it make those? No, I don't know that. 8 This is a model with lots and lots of issues in the underlying 9 data. I could do a diagnostic and pull up the ten thousand 10 observations under each of these and look at them and see what's 11 going on with them, but I haven't done that. You actually have 12 the data so you could do it. 13 Q. I can assure you I can't do it. But let me ask this: Was 14 that because of any uncertainty in the reported data? 15 A. The reason we did the change was because we wanted to model 16 the uncertainty in the reported data and our focus was not on 17 the point estimates, which is what you're looking at, but on how 18 much more uncertainty overall we had in the balances, and, you 19 know, I'm delighted that you're looking at it this way, but this 20 was not our goal. 21 Q. Okay. The long and short of it is, though, that the model 22 itself for reasons that you would have to run a diagnostic on 23 the computer to determine, changed the values of some of the 24 reported numbers, correct? 25 A. It did in all cases do that, yes. Jacqueline M. Sullivan, RPR Official Court Reporter 1030 1 Q. Until we get the 1996? 2 A. That's correct. 3 Q. It didn't change those after that time, did it? 4 A. We did not do any imputations in 1996 and later. 5 Q. Now, in terms of these values, if we take away that 6 comparison but leave the two slides up, do you see down at the 7 bottom of the page -- I'm not going to ask you to read the 8 numbers -- but the concept, that similarly for the years 9 beginning in 1923 that the model adjusted the reported values in 10 disbursements and collections from 1923 through, and if we'll 11 look at the second page of 462, all the way through 1949 except 12 for the disbursements in 1945, correct? 13 A. I'm trying to understand the yellow in 1945. 14 Q. Well, it was saying that's imputed for missing data. In 15 other words, no reported value in the year. If you look over to 16 the left of the charts you'll see 1945 and the -- 17 A. That's the answer, yes. 18 Q. Okay. And again you can't explain why the model changed 19 reported value in those years, can you? 20 A. I don't think that's what I said. 21 Q. Okay. Sitting here today without going and -- 22 A. In terms of a specific number, I cannot. In terms of a 23 process, I have already described it in general, and if you 24 allow me to go in and look at the data, which you all have, we 25 can do more than that. Jacqueline M. Sullivan, RPR Official Court Reporter 1031 1 Q. Let's look at Defendants' Exhibit 462, page three. Let's 2 blow that up where it sits by itself. There we go. Blow up 3 right at the top 1968. 4 Do you see where in 1968 now it is showing that this 5 model imputed a missing value for that year in collections of a 6 hundred million dollars? 7 A. I do, I do. 8 Q. And let's look at the prior page of this Defendants' 9 Exhibit 462, which would be page two? 10 Down at the bottom let's look at 1965 through 1967. 11 And do you see where in 19 -- well, you see 1964 it's $100.2 12 million. In '65 it's $93.4 million. In '66 it's $95.4, and in 13 1967 it's a hundred million. Do you see that? 14 A. I do. 15 Q. And if I told you that there was an audit report that 16 states in -- that for 196 -- as of 1968 that collections in the 17 IIM system were averaging $121 million, what adjustments, if 18 any, would you make? 19 A. Well, if I had the report I would of course have used it. 20 And we would have adjusted this data to conform with that if we 21 found it reliable. 22 Q. Okay. And if Dr. Angel considered it to be a reliable 23 statement of the average collections up to the 1968, for some 24 period you would accept that, correct? 25 A. Oh, I would have accepted that, yes. I would have checked Jacqueline M. Sullivan, RPR Official Court Reporter 1032 1 it, but I would have accepted it. 2 Q. And you would have checked it by doing what, sir? 3 A. Get into the actual report itself and understand it, 4 because often times are these subtle definitional issues, and 5 remember we're different, we think differently here, and there 6 are many different disciplines in this room right now, and I 7 apologize to all those who don't think the way I do, but I'm 8 glad that you're helping me explain myself better, but we're 9 thinking differently, even Dr. Angel and I, who have worked 10 together for a long time. 11 Q. With the information that Dr. Angel provided to you did you 12 ask him to provide you with the backup documentation for that 13 information? 14 A. For this particular year you're talking about, no, we did 15 not. 16 Q. Oh. I'm not saying he provided you any information about 17 this year, but for the information that he did provide to you, 18 did you ask him for documentary support for each of those 19 figures he provided to you? 20 A. Let me go back and raise another question. The answer to 21 your questions is yes, we did every year that Dr. Angel provided 22 me before 1972 we had full documentation for and confirmed. 23 This I think is a -- this is a -- what is it, 1960? What is the 24 year we're looking at here? What is the year we're looking at 25 here, '68? Jacqueline M. Sullivan, RPR Official Court Reporter 1033 1 Q. 1968 was the year. I started asking you questions about 2 it. Did you personally look at any of that documentation or did 3 you have your staff do it? 4 A. I looked at some of it, but no, I didn't do all of it at 5 all. 6 Q. How much of it did you look at? 7 A. Fifteen percent, twenty percent; something like that. 8 Q. If you pull up Defendants' Exhibit 72 and go to page four. 9 Your Honor, I know you've seen this many times, but in 10 light of the witness' testimony I'd like to show it to him. 11 I'm sorry. Let's go back to page one so we can 12 orient, Dr. Scheuren. This is an audit report from you see 13 March 1969 on it? 14 A. Not yet. You have to bring it up for me if you want me to 15 see it here. I do see March 1969. Thank you. 16 Q. Can you see March 1969 now? 17 A. Yes, sir. 18 Q. Let's go to page four, and if you'll bring up the 19 highlighted information there. All right. Let me let you look 20 at that. 21 A. I see that. 22 Q. And it says, Cash receipts are running at the rate of 121 23 million per year. Do you see that? 24 A. I do. 25 Q. Tell me how you would factor that into your analysis. Jacqueline M. Sullivan, RPR Official Court Reporter 1034 1 A. Well, I'd like to get behind that number because in many of 2 these reports there are statements that are not net necessarily 3 made by people close to the system. They're using overall 4 averages or impressions. But if it was a real number, after I 5 had done that review then I would have wanted to have that 6 average be met in the data. 7 Q. Okay. And the average that we've seen in the data for the 8 period coming up, that would be something between, say, probably 9 about ninety and a hundred million dollars, correct? 10 A. It's a little over a hundred in a couple years, but yes, 11 it's under this number considerably. 12 Q. And so that you would want to go then go in and put as a 13 reported value $121 million for a certain number of years 14 leading up to 1968? 15 A. I would have tried to find out what period this was in 16 reference to, but remember I have to change the disbursements 17 too. I can't just change the collections because if this number 18 is -- if the numbers I have about collections are too low and 19 they get raised, then I have to raise the disbursements too or I 20 have to have an explanation for why I can't raise the 21 disbursements. 22 Q. Do you remember that we saw the year 1945 where you had a 23 reported value of collections and no report on disbursements? 24 A. I do. 25 Q. We just looked at that. Jacqueline M. Sullivan, RPR Official Court Reporter 1035 1 A. I do. 2 Q. So for 1968 and the prior years, you could enter a value 3 for collections without putting a value in for disbursements, 4 correct? 5 A. I could have imputed it, yes. 6 Q. You could have? 7 A. I could have left it blank, but if I had the numbers, if I 8 had both numbers, disbursements and collections, I would have 9 gone to Ed and asked him for his advice about what to do. 10 Q. And go back to the first page of this exhibit and blow up 11 the title at the top. 12 And do you see that this was done by the Office of 13 Survey and Review for the Department of Interior? 14 A. Yes. 15 Q. And you would expect them to be knowledgeable of the IIM 16 system? 17 A. I would. 18 Q. And this is a change that we look at here today that you 19 could make to your system still, couldn't you? You could go run 20 it, go back to the office and run that this evening, right? 21 A. This evening maybe. Certainly we could run it quickly 22 after we had confirmed it with discussions. 23 Q. Another area where there was some examination about the 24 accuracy of the information that was used by Ms. Herman was in a 25 period that you talked about, and I think you've talked about it Jacqueline M. Sullivan, RPR Official Court Reporter 1036 1 as the CD&L information that was a period between 1972 and 1985, 2 correct? 3 A. Correct. 4 Q. And what I understood you to say is that as a result of 5 hearing that testimony you did some further analysis about the 6 missing information on the spreadsheets that were used to 7 support her numbers and therefore you prepared a new histogram 8 the day that we looked it; is that correct? 9 A. Let's me say it a little bit differently. I already 10 commended you and thanked you for pointing those out, and when 11 we went back in and looked at that as you pointed it out to us 12 we did not change the histogram itself. I used the histogram to 13 make an assessment. This is obviously somewhat perhaps 14 incomplete to move the upper bound, the 95 percent upper bound, 15 to a higher value in order to deal with the fact that there was 16 more uncertainty in the system than I had expected. 17 Q. And when you're talking about more uncertainty in the 18 system that you expected, you were talking specifically about 19 that 13- or 14-year period, correct? 20 A. That's correct. 21 Q. You said that you got Dr. Angel to provide you with backup 22 documentation for all the information he gave to NORC. Did you 23 have Ms. Herman likewise provide the supporting information for 24 all the data and the reported value she gave to you? 25 A. Generally, no. Jacqueline M. Sullivan, RPR Official Court Reporter 1037 1 Q. Why the difference, sir? 2 A. We had been looking at the '72 forward data for a long 3 time. It appears in one of the earlier reports. We thought we 4 knew what we were looking at, and we've been following the 5 changes in the micro data that Ms. Herman and her team has been 6 introducing over the years in the ways that she has described 7 here, so we thought we knew who we were getting. 8 Q. In light of the uncertainty in the system I think as you've 9 referred to it, that you heard regarding the CD&L period, you 10 think that the upper bounds of the competencies at this level 11 that should be used is 97.5 percent? 12 A. That is correct. 13 Q. Why wouldn't it be 99 percent, sir? 14 A. It could be 99 percent. I indicated in a conversation that 15 you heard, bringing the judge into this, that is a decision that 16 the decision-maker makes as to what that upper bound should be. 17 Q. Okay. To help in that decision, can you explain you would 18 say 97-and-a-half percent rather than 99 percent? 19 A. Well, I had looked at the data, and the amount of 20 uncertainty added to it did not seem to be very large. I have 21 not had a chance to rerun everything, so I'm basing this on just 22 simply my judgment here, and 97 percent of them had seemed to be 23 a good compromise between '95 and '99. 24 Q. Do you recall in your direct testimony saying words to the 25 effect that uncertainty has to be scored in the plaintiffs' Jacqueline M. Sullivan, RPR Official Court Reporter 1038 1 favor? 2 A. I do remember saying that, yes. 3 Q. If I ask you to apply that same principal, would that point 4 toward using a 99 percent confidence level rather than a 5 97-and-a-half percent? 6 A. I've already told you that I moved from 95 to 97-and-a-half 7 because I felt that it was appropriate in the context of what I 8 said about scoring things. I don't think it goes to 99, but 9 it's not my decision. It's the judge's decision. 10 Q. Now, let me ask this: If in fact some of the information 11 that has been used in your system reported value as deemed to be 12 so uncertain that it should not be used as a reported value, 13 what effect, if any, does that have on the upper bounds of the 14 confidence factors? 15 A. That is a question I can't answer without doing it both 16 ways, and if I do it both ways I would expect -- and this is I 17 think where you're going with this -- I would expect that 18 treating it as missing incompletely would increase the 19 uncertainty level to treating it as partially reported, which is 20 the way we're doing it with most of the data here that is 21 reported. 22 Q. And that would cause the upper bound to be a higher total 23 amount, correct? 24 A. That's correct. And we had various looks at what if we 25 were to impute all the data in the data process going from where Jacqueline M. Sullivan, RPR Official Court Reporter 1039 1 we are now into the future. There's a way to do that. We 2 haven't done that. One of the things that repeatedly you've 3 heard with other witnesses is that with me more could be done, 4 and we are happy with this analysis given the time that we were 5 allowed to look at it and do it, but with more time we would do 6 more. 7 Q. Certainly you're not happy with this analysis in light of 8 some of the things that you've seen today with understatement of 9 revenues in between the 1930s and '40s and the 21 million 10 average leading up to 1968, are you? 11 A. I am not happy with the fact that we didn't incorporate 12 some of what you've told me today into this. 13 Q. Okay. Let's pull up Plaintiffs' Exhibit 137. 14 A. Other should I say we'd have to do. 15 Q. Put it at the top so we can see the title. 16 Do you see that this is a page that appeared in 17 Statistical Methods and Medical Research in 1999 by professor of 18 statistics at Penn State by the name of Joseph Schaeffer, do you 19 see that? 20 A. Yes. 21 MR. WARSHAWSKY: Do you have a copy of it? 22 THE COURT: You're looking at it on the screen. 23 MR. WARSHAWSKY: Normally they've been providing 24 copies. 25 MR. DORRIS: I don't, but why don't you take mine? Jacqueline M. Sullivan, RPR Official Court Reporter 1040 1 MR. WARSHAWSKY: I don't want to take your marked-up 2 copy. 3 MR. DORRIS: It will be helpful to ask you to follow 4 along. 5 MR. WARSHAWSKY: Maybe. 6 MR. DORRIS: I know what I want to ask about this one. 7 BY MR. DORRIS: 8 Q. Professor Schaefer is someone that you know and works in 9 and is knowledgeable in the field of imputation, correct? 10 A. Yes. I know him quite well. I'm mentioned in his book. 11 I'm mentioned in a lot of books. 12 Q. I made it through the first two pages of this article 13 before I couldn't follow it anymore. He's written here you say 14 a primer for people on multiple imputation. Do you see that? 15 A. His model is what we're using for MPI analysis. 16 Q. I want to concentrate on this paragraph that runs over to 17 the next page. 18 If you will blow this up. 19 And he refers to multiple imputations, MI. Do you see 20 that? 21 A. MI is not the only principal method for handling missing 22 values, nor is it necessarily the best for any given problem. 23 Q. That's entirely consistent with what you've testified to? 24 A. Yes, I agree with that. The word "principal" means that 25 you know statistically the principal. Jacqueline M. Sullivan, RPR Official Court Reporter 1041 1 Q. If you want to read the in between be my guest, but I want 2 you to go to the highlighted part that begins down at the bottom 3 of the page. It says, Given sufficient time and resources one 4 could perhaps derive a better statistical procedure than MI for 5 any particular problem. In real life applications, however, 6 we're missing data are a nuisance rather than a major focus of 7 scientific inquiry. A readily available approximate solution 8 with good properties can be preferable to one that is more 9 efficient but problem-specific and complicated to implement. Do 10 you see that? 11 A. Yes, I do. 12 Q. Is multiple imputation, should it be used to impute values 13 for the missing data where they are a nuisance other than a 14 major focus of the inquiry? 15 A. Can I answer that question by interjecting something first 16 before I answer it? 17 Q. Well, if you can answer it yes or no and then explain your 18 answer would be the preferable approach. 19 A. That's fine. I don't think that using multiple imputation 20 only when you're going to get rid of nuisance problems as needed 21 is appropriate. You can use it for other methods too, but there 22 are other specialized methods that could be used. I think it 23 could in the mathematical sense dominate. They could be better 24 if you had enough time and money. 25 Q. So you would not agree to the extent that Professor Jacqueline M. Sullivan, RPR Official Court Reporter 1042 1 Schaeffer is implying that it should be used where missing data 2 is a nuisance rather than the major focus of scientific inquiry, 3 you wouldn't agree with that? 4 A. I couldn't agree with it, no. 5 Q. Now, let me ask you to look at your article. Do you have 6 that? It was passed out today. It's Defendants' Exhibit 507. 7 I suspect you have a copy with you, Dr. Scheuren. 8 A. Am I allowed to elaborate on what you just said? 9 Q. You can answer any questions that the government asks you. 10 A. Go ahead then. 11 Q. Looking at your article -- I'm going to pause just for a 12 moment till it's brought -- you need to switch over, I think. 13 Sorry to do that to you. 14 I'll tell you what I am working with now. 15 Go to the third page of this exhibit, please, and go 16 under 4.2, the second paragraph, and blow that up for me. 17 The part I'd like you to look at and I'd ask to 18 highlight is a sentence that begins about halfway down there and 19 says "I still remain skeptical of models, however, and worry 20 about people complaints believing them too much." 21 Can you highlight that statement for me, please, sir? 22 Exactly. 23 Do you see that? 24 A. I do. 25 Q. What were you saying about that? Jacqueline M. Sullivan, RPR Official Court Reporter 1043 1 A. Let me give you some background, if I might. In the world 2 of sampling that I'm in, one of the greatest statisticians, who 3 recently died, W. Edwards Demming, did not believe in models. 4 He thought that we should base our inferences on what are called 5 frequentious-based as the same from Bayesian methods. And 6 frequentious-based methods are what we usually do when we're in 7 the world I live in, in the sampling world, because of Demming, 8 and we, because of Demming, did not adequately pay attention to 9 the fact that we really needed to use models for a whole lot of 10 settings that we weren't using them for, and where it's 11 particular when there are missing and misreported observations. 12 Even Demming would have agreed that we need missing data models 13 for missingness, but to the extent to which you have only one 14 model is a problem, because you really need to look at multiple 15 models because you need to look at the fact that the models 16 themselves have concerns with them, and we've been doing that 17 here, and because of the way this is being done, we're 18 presenting a model without all the models. 19 Q. And you really have looked at one model; is that correct? 20 A. No. You've actually called my attention to something we 21 gave you earlier, which is we've looked at at least two models, 22 and you asked me about the difference between those two models. 23 We spent quite a bit of time on that. There were other models 24 we looked at too. 25 Q. Two models constructed or run in the same analytical Jacqueline M. Sullivan, RPR Official Court Reporter 1044 1 framework, correct? 2 A. The word "model" and the word "analytical framework" phrase 3 are not equivalent. 4 Q. Let me ask you to step back, and there's a paragraph on the 5 other column just above where you've highlight there that begins 6 the non-ignorable case. Do you see that sentence that begins, 7 The non-ignorable case NMAR which adds variance with intractable 8 bias is a generally have-to-live-with form of missingness? 9 A. That's correct. 10 Q. Can you state that in more layman's language? 11 A. There are three types of missingness that are described in 12 here by the people at Don Rubin in 1976. One kind is just have 13 data that is not there. There's nothing particularly bad about 14 it except that your sample is too small and you don't have as 15 much data as you thought. There's nothing you need to do about 16 it except not over-react to it. The second phase which is 17 missing at random is a case where if you know what's missing and 18 how to fix it then you can adjust the data so that it gives you 19 answers that are not biased, but if you don't have enough 20 information from the data set to deal with certain things, then 21 you cannot adjust for the missing -- the non-ignorable portion. 22 Because the non-ignorable portion is usually something that you 23 don't know, not in the data set, somewhere else, and maybe not 24 on anyone's radar screen, and that is affecting the results. 25 Sometimes you can't even tell that it's there, and in this sort Jacqueline M. Sullivan, RPR Official Court Reporter 1045 1 of philosophical discussion, which is what this is, and 2 recollection, you need to be cognizant of that. 3 Q. And the missing data that we're confronted with in this 4 matter is which of the three types? 5 A. It's all three. 6 Q. Okay. 7 A. I've been talking about the all three part for quite a 8 while. 9 Q. Let's look at the next page. There is one last quote I 10 want to ask you about in this article. Under 6.1, if you would 11 pull that up and highlight that in the first sentence, it's kind 12 of a long sentence, you see there you wrote, Typically in my 13 practice subject matter experts believe in their imputation 14 models more than is warranted, possibly in part to the chronic 15 problem we have had in many surveys of underestimating 16 imputation variance and expense taken in time and money in 17 adjusting for missingness. Do you see that? 18 A. Yes, I do. I think it applies in this case. 19 Q. How does it apply in this case? 20 A. I think Professor Cornell believes that about his model. 21 Q. Okay. 22 A. It doesn't have any measure of uncertainty. 23 Q. So what I hear you saying, because you recognize that there 24 is tremendous uncertainty here -- well, you may not like that 25 word, tremendous. Would you agree there is tremendous Jacqueline M. Sullivan, RPR Official Court Reporter 1046 1 uncertainty in the data here? 2 A. A little more quantative and less qualitative, but yes, 3 I'll let you get away with that. You're doing a good job, by 4 the way. Thank you. 5 MR. DORRIS: There's one thing we can agree on, your 6 Honor. Civility here. The questioner and the witness can 7 stipulate to one thing. I'm sorry for the interruption there. 8 BY MR. DORRIS: 9 Q. Okay. Dr. Scheuren, you used a phrase in your direct 10 examination that I didn't understand. You said that the 11 statistician need to let the data speak to them. 12 A. That's correct. 13 Q. What do you mean? 14 A. Well, it means a simple as -- you used this illustration 15 earlier as doing chatter blots with the data as relating the 16 data you have with other things you think are true, okay, or 17 with experts who may have created the data or who have knowledge 18 of what it means and all of that you factor into your 19 understanding of what the data is and allowing it to speak. 20 Q. For example, of what we went through before lunch when I 21 showed you the collection and disbursement data from 1909 to 22 1911 and we looked at what the rate was there and then looked at 23 how the rates before and after were different than that, do you 24 remember us going through that? 25 A. Oh, I certainly do. It was very well done. Jacqueline M. Sullivan, RPR Official Court Reporter 1047 1 Q. And that would be something that a statistician would let 2 that kind of data speak to him, correct? 3 A. Correct. If my objective had been to focus on approving 4 the point estimates, then I would have done a lot more work with 5 that. 6 Q. Okay. Now, let's go to Defense Exhibit 460. It will help 7 make the question clearer. Okay. This is kind of a -- by the 8 way, did you put this Power Point slide together? 9 A. I wrote these words. I wrote these words, yeah. I may 10 have given it to them as a Power Point. I can't remember. 11 Q. You say, After examining the existing data and then using 12 multiple imputation to generate estimates of missing data and to 13 assess missing data uncertainties. Do you see the second bullet 14 there that I was just reading? 15 A. Yes, yes. 16 Q. You testified during your direct examination about 17 something and I didn't follow it, but you did say you did 18 describe the process as how you go about beginning to impute the 19 data in that process and you said something to the extent that 20 you would essentially make a first guess at that and then move 21 on. Can you explain to me what you mean by making a guess at 22 those values? 23 A. Well, you have some kind of set of prior beliefs which 24 you've incorporated and you have this notion we're going to use 25 a multi vari normal distribution in background and then we're Jacqueline M. Sullivan, RPR Official Court Reporter 1048 1 going to do our best to make an estimate from the available data 2 of what the mean and the variance of this data set are. A 3 vector mean and a coprehence (ph) matrix, and we do that, and 4 then we use the results, okay, the distribution we've created, 5 to draw an observation from random numbers, using random numbers 6 from that distribution. 7 Q. Okay. So that there is work done based on certain beliefs 8 that you have that then lead to the selection of the random data 9 to put in for the imputed data, correct? 10 A. I have prior beliefs and I have the data itself, the 11 reported data, and I use those to produce an initial starting 12 point for the multiple imputation, and then I repeat that 13 process over and over again and going back and forth until the 14 process converges, and then I stop and then I do another one and 15 so forth. 16 Q. Is there any way, looking at the information that you 17 provided to us electronically, for us to see these prior beliefs 18 that you have that lead to this first step or this really second 19 step that you're talking about here? 20 A. I think it's embedded in the software that we gave you 21 access to. Seeing the way you've conceptualized it I think we 22 probably would have to do a little more to make it visible for 23 you. 24 Q. And it's not visible? 25 A. It's in the software. It's in the proc MI, the multiple Jacqueline M. Sullivan, RPR Official Court Reporter 1049 1 imputation procedure from SAS has this information in it. We're 2 calling commands from that proc, and I've already mentioned to 3 you that related to what Joe Schaeffer has done. 4 Q. Well, let me ask it to you this way: With what we've been 5 provided so far it's a little bit like you've told us 6 information that you've gone and put into a black box and now we 7 look at what the results are, correct? 8 A. Well, how about opaque? Can you say they're opaque? 9 Q. I'll settle for opaque. Okay. 10 A. That's fair, that's why I have to look at the data, and 11 look at the software in detail. 12 Q. What do you mean by "opaque"? 13 A. We didn't walk you through the process from front to back, 14 you know, connecting the words that you heard me say with the 15 software that you got Friday, and we perhaps should do that. 16 Q. And you've not brought anything into court today that would 17 permit you to do that for us, have you? 18 A. No. As a result of your question about Professor Cornell's 19 data we spent the lunch hour providing that data to you. 20 Q. Well, I've had some interesting lunches during this trial 21 too, so I apologize for taking up your lunch hour, but I do 22 understand from government counsel that that data about his 23 analysis of Dr. Cornell's information can be provided to us, and 24 what I'm going to ask your Honor is that I go as far as I can 25 go, cover everything except that, any questions that may come Jacqueline M. Sullivan, RPR Official Court Reporter 1050 1 from that, that once that's provided we could then ask any 2 follow-up questions, if any, that we have of Dr. Scheuren on 3 that. 4 THE COURT: We'll talk more about that. Go as far as 5 you can and then we'll talk about it. 6 MR. DORRIS: Okay. 7 BY MR. DORRIS: 8 Q. Now, let me ask you this question about the relationship 9 between revenues and disbursements. You explained to me I think 10 before lunch that you would look at that and there would be some 11 type of coefficient or colinear relationship between those two, 12 correct? 13 A. We would look for that, and in fact we found it, yes. 14 Q. And let me ask it to you this way: The revenues are to a 15 large extent driven by market forces, correct? 16 A. Can we use "collection" instead of "revenue"? Whatever you 17 want to do. 18 Q. I'll be glad to use collection. 19 A. Okay. 20 Q. What do you mean by "collection"? 21 A. Well, because there's a lot of things in the IIM system 22 that aren't really revenues, okay? For example, bid deposits, 23 okay, it's not revenue. It's just a deposit. 24 Q. So where you've put "collections" on your chart you've just 25 tried to have those include things that are not the total Jacqueline M. Sullivan, RPR Official Court Reporter 1051 1 revenue but only collections; is that right? 2 A. Collection is the total -- is the thing we're looking at 3 here. What's coming into the system is what we're calling a 4 collection. 5 Q. For example, you gave bid deposits. Are those in your 6 collection number or not? 7 A. They're in the collection number. 8 Q. I'll use collections with you. Now, collections are driven 9 to some extent by market forces, correct? 10 A. Of course. 11 Q. The price of all -- the price of a quart of timber, grazing 12 rights and the like? 13 A. All of that's true. 14 Q. Disbursements are not driven as much by market forces, are 15 they? 16 A. There are different periods in this time period. You know, 17 once -- you know, at some point until you remember what year it 18 is, if you have more than $5.00 in oil receipts you got a $5.00- 19 check and disbursements of, you know, disbursements of other 20 kinds of revenue as well, so once you receive a certain 21 threshold you got the money. In the old times that wasn't 22 always done. 23 Q. Well, let me ask you if this. 24 A. So they're linked, by the way, in modern times because of 25 this rule. Jacqueline M. Sullivan, RPR Official Court Reporter 1052 1 Q. When you say "modern times," what time period begins 2 "modern times" for that answer? 3 A. I know the answer to that but I can't give it to you right 4 now. Sorry. I just don't know. It's been a long time since 5 this change has occurred. 6 Q. Okay. But before that period of time they are not linked 7 as a result of that policy; is that correct? 8 A. That's correct. 9 Q. And so that while your data established a colinear 10 relationship between collections and disbursements, actually 11 market forces affect those two numbers in very different 12 fashions, don't they? 13 A. You just ignored what I said, didn't you? 14 Q. I may not have understood what you told me. 15 A. Part of this time, part of this historical period, that is 16 correct, but although you don't establish that the effect of 17 disbursements, you just said something about revenue. Revenue 18 and disbursements are linked in modern times. In earlier 19 periods they were not necessarily linked. But if you look at 20 the data during the Great Depression in our data set you will 21 see that disbursements exceed collections year after year, and 22 that's perfectly understandable. The money was coming out of 23 the trust in order to feed people. 24 Q. And that's that same period where we've seen even what Dr. 25 Angel and Ms. Herman note, that the revenues, their word was Jacqueline M. Sullivan, RPR Official Court Reporter 1053 1 "revenues," were understated during that same period, correct? 2 A. I'm not sure when the beginning. I remember the 1949 end 3 point, but I don't remember what the early point is. 4 Q. How can there be a colinear relationship between balance 5 information and collections and disbursements when even 6 according to you the balances do not foot with the collection 7 and disbursement amounts? 8 A. Well, there are two things going on in which you've said. 9 First of all, a correlation can exist even though there is not a 10 direct functional relationship between something because there's 11 uncertainty. The extent of the correlation is a measure of 12 that, so of course there can be a correlation between these 13 values, even if they don't foot. 14 Q. Now, there are times in your numbers, for example, where 15 there are known information. 16 Let's pull up Defendants' Exhibit 462 to 19, the 17 second page, and blow up the first four lines or so. 18 Let me just ask you this: You see we're going along 19 and there is a missing value for the balance in 1929. Do you 20 see that? 21 A. Yes, I do. 22 Q. And so you're working with reported values for collections 23 and disbursements in the years before and after where that 24 missing data is. Do you see that? 25 A. Um-hmm. I've analyzed this data. Jacqueline M. Sullivan, RPR Official Court Reporter 1054 1 Q. And you see that it says the balance as of the end of the 2 year, fiscal year 1928, is $74.7 million, correct? 3 A. I do. 4 Q. And then the next year there is about $9.5 million more 5 collected than disbursed, correct? 6 A. Correct. 7 Q. And then your imputation model has put in a balance that, 8 if I did the math right, would be at about $85 million, it puts 9 in a value of $33.3 million there, correct? 10 A. Yes, I know that. I've analyzed this data, as I already 11 told you. I noticed this when I saw it. 12 Q. Okay. So even when the model then comes and imputes 13 balances, it imputes balances that don't foot, correct? 14 A. That's correct. 15 Q. Is that because of the tremendous uncertainty in the data? 16 A. We looked at this data and we actually we had this 17 procedure of looking for outlies, and one of the things that we 18 did with the data that you have here is that we looked at the 19 outlies here too, and this is the worst. There are other out- 20 lies but they're mostly small. This is a spectacularly bad 21 outlier. 22 Q. The one that I just give you? 23 A. That's correct. I'm not surprised given who you are that 24 you would have noticed it, and we noticed it too and we went 25 back and looked at that. Our focus here is not on the balance. Jacqueline M. Sullivan, RPR Official Court Reporter 1055 1 Our focus is here on the collections and disbursements, okay, 2 and so we didn't like it but we left it. 3 Q. You didn't like it but you didn't do anything about it, 4 correct? 5 A. Correct. 6 MR. DORRIS: Your Honor, I don't have any further 7 questions subject to the one note I mentioned about the analysis 8 that the government has agreed to provide -- or the data they've 9 agreed to provide to us. 10 THE COURT: Let's talk for a minute about that. Mr. 11 Warshawsky, do you have something to say about it? 12 MR. WARSHAWSKY: Yes, your Honor. 13 THE WITNESS: Can I say thank you? 14 MR. DORRIS: Sure. 15 THE COURT: And you may also step down. Thank you, 16 sir. 17 MR. WARSHAWSKY: Your Honor, we have what the 18 plaintiffs have asked for, and it's provided to them. It 19 consists of two files. One is actually the SAS application I 20 believe that they already had. The other is an Excel file. 21 This was an attempt to reconstruct -- 22 THE COURT: Dr. Scheuren, actually, would you come 23 back up? I have a couple of questions I'd like to ask you. 24 THE WITNESS: Oh, certainly sir. 25 THE COURT: Go ahead. Jacqueline M. Sullivan, RPR Official Court Reporter 1056 1 MR. WARSHAWSKY: It's an attempt to reconstruct Dr. 2 Cornell's model. We've never been provided any of the data or 3 information regarding Dr. Cornell's model. We first learned of 4 it, it was actually Wednesday, last Wednesday when he testified. 5 We had the good fortune that I would say that his model 6 apparently didn't change since the March 2008 brief that 7 plaintiffs filed. He made no adjustments even though we had 8 provided information in our brief, but although Thursday we 9 were, told both sides had been told to provide this information, 10 we still haven't received anything about Dr. Cornell's model, so 11 that's what we've got, is an attempt to reconstruction Dr. 12 Cornell's efforts and NORC's conclusions and now plaintiffs have 13 it. I'm certainly prepared to do the redirect on everything but 14 this and we can have Dr. Scheuren come back tomorrow if that's 15 the Court's wishes. 16 THE COURT: You're turning over the SAS application 17 and the spreadsheet? 18 MR. WARSHAWSKY: It's an Excel document, and Dr. 19 Scheuren could probably tell you about it, but basically my 20 understanding is it's the inputs that go into the application. 21 THE COURT: I thought I heard Mr. Dorris asking more 22 than that. I think he wants to look inside the opaque box. 23 MR. WARSHAWSKY: Well, your Honor, I'm glad you asked 24 about the opaque box. The opaque box is a reference to the SAS 25 application. I advised Mr. Smith before we had any discussion Jacqueline M. Sullivan, RPR Official Court Reporter 1057 1 with the Court, I should say before we had the change, I advised 2 him Thursday that the application was a multiple imputation 3 application, and in fact when we delivered it on Friday I told 4 him that we had somebody from NORC waiting by if they needed any 5 assistance walking through it. That was in the afternoon. I 6 heard nothing as of 9:45 that evening. Nothing. Saturday I 7 spoke with Mr. Smith. He called simply to inquire about Dr. 8 Angel's information, so they've never asked us for the 9 assistance that we offered, and it's frankly a little odd at 10 this point to hear plaintiffs complaining that they haven't been 11 walked through the SAS application because the offer was there 12 as early as Friday afternoon. 13 THE COURT: All right. Do you want to do some 14 redirect now? 15 MR. WARSHAWSKY: I can do the redirect, your Honor, on 16 everything, obviously reserving -- 17 THE COURT: Actually I want to ask two or three 18 questions and then you can do your redirect, so I released you 19 prematurely, Doctor. 20 I need to understand better than I do what this ten 21 thousand iterations business is all about. I don't quite 22 understand what you mean by that. Do you mean ten thousand 23 pieces of data, do you mean you run the model ten thousand 24 times, and if you run it ten thousand times how does the 999 25 differ from the thousand and so forth? Jacqueline M. Sullivan, RPR Official Court Reporter 1058 1 THE WITNESS: We have a knowledge that we have from 2 the data to begin with and we use that to start the modeling 3 process. We bring it in. We then use random numbers to 4 actually do the simulations so each simulation is different 5 because of the randomness that we're adding, but they're 6 constrained by what we know about the model, so after we do the 7 first iteration we construct the posterior from the data. And 8 we compare the results, the parameters that we're interested in, 9 which was usually the means and variance, covariances in this 10 world, we compare them to what or what our starting values were, 11 and if they're not close, and close is really quite close, if 12 they're not close we average the first iteration with our 13 estimate, our starting value, and then we do it again and we 14 keep doing that until the numbers converge, so we get estimates 15 of the means and totals, means variances that are the same from 16 one iteration to the next, and this goes on a long time. Of 17 course it's microseconds because it's in the computer, and that 18 is the first iteration, that is the first imputation that we do, 19 and then we start begin, okay, a new starting point and do the 20 process over and over again until we've done ten thousand of 21 those. That's how we did it. 22 THE COURT: If I were to sit down at a keyboard with 23 that software would it show me what you mean? 24 THE WITNESS: Yes, yes. You have to have SAS on your 25 computer and things like that, but yes. Jacqueline M. Sullivan, RPR Official Court Reporter 1059 1 THE COURT: The other thing I want to talk about is 2 the use of the Osage average share as one of the five variables 3 because you've got nothing about a year and an Osage variable. 4 Now, is that variable useful because of the quantity of it, or 5 is it useful because of the comparative size to this year's to 6 next year's to the year after that, or does it have anything to 7 do with oil price versus timer price? 8 THE WITNESS: I can't speak to oil price versus timber 9 price. It does give us some information about what's going on 10 with the economy generally, and of course it says specifically 11 regarding oil, and it goes all the way back into this period, 12 and it is directly relevant to the Indian trust system. We 13 could have added other variables. 14 THE COURT: Like what? 15 THE WITNESS: Price indices, GDP, you know, general 16 economic variables, but again we're talking about very rural and 17 regional areas where these general kinds of variables might not 18 get us down from enough or have any more predictive value than 19 the Osage did. 20 THE COURT: You said at one point, I think, you could 21 have run this without the Osage numbers. 22 THE WITNESS: We have done that, yes. 23 THE COURT: What's the result? How sensitive is this 24 model to using the Osage? 25 THE WITNESS: I can't give you the answer to that Jacqueline M. Sullivan, RPR Official Court Reporter 1060 1 right now, but it is sensitive in the early period, before 1909. 2 It is sensitive. It's not going to make much difference. 3 THE COURT: If you ran it with the Osage numbers you'd 4 get what, higher balances or lower calculated balances? 5 THE WITNESS: I'm not sure what I would do if I 6 were -- I don't know the answer to that question, your Honor. 7 If I were really pressed and said you can't use Osage because 8 it's this, that or the other thing, then I would probably go 9 look for some other variables which I didn't think was 10 particular relevant, but I would check and maybe find them to be 11 relevant. 12 One thing that's been said a lot is we're operating 13 quickly here. If we had more time, you heard me make a comment 14 just a little while ago about an outlier that was identified by 15 plaintiff, the Plaintiffs' lawyers, which we identified 16 ourselves but after we delivered this. And of course I would 17 have fixed that, okay, but I don't think it would make much 18 difference so I didn't bring it up, but we've looked at all the 19 outliers. If we had more time we'd do better, but I think the 20 basic results that we have -- 21 THE COURT: You're comfortable with them? 22 THE WITNESS: With the terms of the overall 23 uncertainty. I mean, we're looking at something -- there's a 24 lot of uncertainty here and the uncertainty cuts against the 25 government, but the kind of uncertainty that is on the table Jacqueline M. Sullivan, RPR Official Court Reporter 1061 1 here is rather different in size than the numbers that you've 2 seen from the plaintiffs by a factor of ten. 3 THE COURT: Yes. Okay. Thank you. 4 Do you want to do some redirect, Mr. Warshawsky? 5 THE WITNESS: Thank you, your Honor. 6 MR. WARSHAWSKY: Thank you, your Honor. I just have a 7 couple of redirect. 8 REDIRECT EXAMINATION 9 BY MR. WARSHAWSKY: 10 Q. Dr. Scheuren, I'd like to ask you about a couple of things 11 that Mr. Dorris inquired about with respect to information you 12 received from Morgan Angel. 13 A. Yes. 14 Q. Now, you may recall on cross you were being asked about 15 some of the collection figures from 1922 to 1949. 16 A. Yes. 17 Q. Do you remember having any conversations with anybody from 18 Morgan Angel about disbursements during those periods? 19 A. No. 20 Q. If the disbursements -- let me ask it this way. If the 21 disbursements were similarly understated, there is a discussion 22 about collections being understated, how would that impact your 23 calculated balance estimates? 24 A. It wouldn't have changed the balances potentially at all. 25 The things we saw going forward, especially in the period Jacqueline M. Sullivan, RPR Official Court Reporter 1062 1 before, the early period, is that even though it doesn't have to 2 move together, as I've stipulated, it tended to move together 3 up or down, except during the great economic stress during the 4 Depression. 5 Q. And just to be clear, when you're talking about things 6 moving together, you're talking about the collections and 7 disbursement? 8 A. Yes. If collection was up disbursement would also go up 9 and vice versa. 10 Q. And your focus is the calculated balances? 11 A. That's what we're after, yes. We're not trying to make 12 best estimates for each of these cells. 13 Q. Now, if you could pull up Defendants' Exhibit 72, please, 14 page four. Okay. And if you could highlight the section that 15 Mr. Dorris asked about here. 16 Do you remember the discussion about cash receipts are 17 running at a rate of $121 million per year? 18 A. Yes, I do. 19 Q. Do you know if Dr. Angel actually used that figure in his 20 analysis? 21 A. I don't know. I don't think he did, but I don't know. 22 Q. You weren't present this morning when Dr. Angel explained 23 on his redirect why he didn't use that, were you? 24 A. No, I was not. 25 Q. So you don't know the reasons that he rejected that figure? Jacqueline M. Sullivan, RPR Official Court Reporter 1063 1 A. No, I don't. 2 I want to make this point. I asked for data from Dr. 3 Angel that was connected directly to a historical, you know, 4 historical document, okay. And that's what we got and his 5 judgment is what I relied on. 6 Q. Let's pull up Plaintiffs' Exhibit 137. I'm not sure, do we 7 need to get that from you? Yes. 8 Dr. Scheuren, Mr. Dorris asked you about a paragraph 9 that continued from page one to page two. If it assists you I 10 can provide you with Mr. Dorris' copy. 11 A. Let me have the copy. Just looking through the lines it's 12 a little strange. It's a missing data problem. 13 Q. Okay. All right. And on the screen we do have those first 14 two pages up. Now, do you remember your testimony about that 15 paragraph? 16 A. Yes, I do. I wanted to elaborate and I didn't get to do 17 that, but maybe I can do it now. Maybe you'll ask me a question 18 and I can do it. 19 Q. Here's your chance. Why don't you agree with that 20 statement, Dr. Scheuren? 21 A. Well, I think we're having a situation where there is a lot 22 of different approaches to missing data. And Joe is trying to 23 convince people that at least in some cases multiple imputation 24 is the method of choice but he doesn't want to be absolute about 25 it and say it's always the method of choice. That doesn't make Jacqueline M. Sullivan, RPR Official Court Reporter 1064 1 friends and influence people so he's careful and he's modest in 2 his statement. I actually believe that is much more of an 3 advocate for multiple imputation than that, but he's trying to 4 deal with people who -- and there's some controversy among 5 multiple imputation. If you looked a little bit at the material 6 that I had in the paper that we looked at this morning, all you 7 Google when you Google -- I'm very sorry, Jackie -- you Google 8 this issue, the first thing that comes up is a paper by a good 9 friend of mine, Bob Fay, that has some problems with multiple 10 imputation, so there's some underbrush here. However, that kind 11 of underbrush actually leads to a particular kind of problem 12 which is the opposite of the problem here. If there is a 13 problem with the multiple imputation in this particular setting, 14 it doesn't affect our result except that we may have slightly 15 overstated the uncertainty as a result of it. Slightly 16 overstated. 17 Q. And what is the significance if you overstate the 18 uncertainty in this case? 19 A. That the amount of money that is at issue here is larger, 20 but we've already done that and I think I'm weighing all these 21 things, and what I heard the other day from Michelle Herman and 22 her response from the Plaintiffs' lawyers, I weigh that in with 23 the fact that we may slightly have overstated, but I don't think 24 that's very important in this case and I think that a modest 25 adjustment to a 97.5 upward bound is the right answer here in Jacqueline M. Sullivan, RPR Official Court Reporter 1065 1 rough justice. 2 Q. And to the extent you overstated, does that inure to the 3 benefit of the plaintiff or the defendant? 4 A. It benefits the plaintiffs. It benefits the plaintiffs. 5 MR. WARSHAWSKY: Your Honor, I have no more redirect 6 subject to whatever is covered. 7 THE COURT: Doctor, I think we're going to release you 8 today. I think we're going to have to say that you're subject 9 to recall, but when and under what circumstances remain to be 10 seen. 11 And we're going to take a ten-minute break. 12 MR. WARSHAWSKY: One request, your Honor. I know Mr. 13 Scheuren is going out the country I think next week. 14 THE WITNESS: Yes. 15 MR. WARSHAWSKY: So we request that he be called this 16 week. 17 THE WITNESS: Thank you, Judge. 18 COURTROOM DEPUTY: Court stands in recess. 19 (Recess taken at about 3:17 p.m.) 20 COURTROOM DEPUTY: Please come to order and be seated. 21 THE COURT: Okay. What's next? 22 MR. WARSHAWSKY: Your Honor, just to clarify, as I 23 indicated, we would like to get Dr. Cornell's backup data as 24 well and his analysis, and hopefully that's coming soon. I 25 haven't heard plaintiffs clarify that one. Jacqueline M. Sullivan, RPR Official Court Reporter 1066 1 Also, if we could get an indication as to when Dr. 2 Scheuren is to be recalled so we can make sure he's here. 3 THE COURT: Does anybody want to respond to any of 4 that? Mr. Dorris? 5 MR. DORRIS: If Mr. Warshawsky had asked me about it 6 at the break I would have been glad to address that with him. I 7 don't know what backup from Dr. Cornell he wants. We gave him 8 all of the information with the documents supporting it on March 9 19 and there hasn't been further analysis. Will there be? 10 We've indicated to Mr. Warshawsky we think there will be and 11 that we'll provide that to him as soon as it's done and that's 12 what we're committed to doing, but that has not -- that process 13 is not done and we don't have anything further to provide to him 14 at this point. 15 THE COURT: Okay. What about Scheuren, when do you 16 want to bring Scheuren back? 17 MR. DORRIS: Your Honor, they provided information, 18 and I would think given the witnesses I'm responsible for 19 tomorrow that Thursday would probably be better. Today is 20 Tuesday. Thursday would probably be best for Dr. Scheuren. 21 That gives me tomorrow to look at it. 22 THE COURT: Let's ask Dr. Scheuren to be available on 23 Thursday. 24 MR. WARSHAWSKY: Very good, your Honor. 25 THE COURT: Mr. Siemietkowski? Jacqueline M. Sullivan, RPR Official Court Reporter 1067 1 MR. SIEMIETKOWSKI: Your Honor, the defense calls 2 Terence Kehoe. 3 THE COURT: All right. 4 COURTROOM DEPUTY: Please raise your right hand. Do 5 you solemnly swear that the testimony you give to the Court in 6 the case now on trial will be the truth, the whole truth and 7 nothing but the truth so help you God? 8 DR. KEHOE: I do. 9 COURTROOM DEPUTY: Thank you. Please be seated. 10 MR. SIEMIETKOWSKI: Your Honor, we intend to have Dr. 11 Kehoe testify regarding two areas. Primarily Dr. Kehoe will 12 testify regarding IIM banking policy, specifically where he has 13 been throughout the history of the IIM system. Secondarily and 14 more briefly, we expect Dr. Kehoe to testify in terms of the 15 Plaintiffs' calculation of alleged benefit to the government. 16 Now, your Honor, Dr. Kehoe has never testified as an 17 expert witness before. In fact, he has never testified at all. 18 We would like to qualify him as an expert like Dr. Angel in the 19 history of federal Indian relations and in the federal Indian 20 records. To do that, however, rather than simply show the Court 21 a C.V., we'd like to do that, but also lay the foundation with 22 Dr. Kehoe. 23 THE COURT: Fine. Go ahead. 24 MR. SIEMIETKOWSKI: And I ask that DX 498 be put on 25 the screen, please. Jacqueline M. Sullivan, RPR Official Court Reporter 1068 1 TERENCE KEHOE, Ph.D, WITNESS FOR THE GOVERNMENT, SWORN 2 DIRECT EXAMINATION 3 BY MR. SIEMIETKOWSKI: 4 Q. Dr. Kehoe, would you please state your full name? 5 A. Terrence Patrick Kehoe. 6 Q. Where do you work? 7 A. I work at Morgan Angel & Associates in Washington, D.C. 8 Q. What's your profession? 9 A. I'm a historian. 10 Q. Dr. Kehoe, would you please explain your educational 11 background? 12 A. I have a bachelor of science degree in business 13 administration from Bowling Green State University and a 14 master's and a Ph.D in American history from Ohio State 15 University. 16 Q. In the course of your formal studies have you had a chance 17 to learn the conduct of historical records research? 18 A. Yes. In graduate school I had extensive experience 19 researching public policy and how it changed over time working 20 in various state and federal archives primarily in government 21 records of various kinds. 22 Q. What about your employment, Dr. Kehoe, have you ever taught 23 at all at the university level? 24 A. Yes. I was a visiting professor at Wake Forest University 25 for like two years, and among the courses that I taught there Jacqueline M. Sullivan, RPR Official Court Reporter 1069 1 was American environmental history, history of the American 2 West, the United States since 1945. 3 Q. Did any of those courses taught at Lake Forest University 4 touch about Indian matters? 5 A. Certainly the course on environmental history did, and most 6 extensively the course on the American West. That was a big 7 component in that course. 8 Q. Dr. Kehoe, how long have you worked at Morgan Angel? 9 A. I joined the firm in December of 1999. 10 Q. Who has been your supervisor there since December of 1999? 11 A. Dr. Edward Angel. 12 Q. Could you provide the Court an estimate, please, of what 13 percentage of your work at Morgan Angel has been focused on 14 Indian matters? 15 A. About 75 percent. 16 Q. And what about the other 25 percent? 17 A. Primarily devoted to research and writing as it relates to 18 environmental litigation. 19 Q. Within Indian matters, what types of issues have you 20 focused on? 21 A. Well, I've done some research and writing concerning 22 natural resources management on reservations. I would say that 23 the bulk of my work has concerned the management of both tribal 24 trust funds and individual Indian monies. 25 Q. While employed at Morgan Angel who have your clients been? Jacqueline M. Sullivan, RPR Official Court Reporter 1070 1 A. The work that I have done involving American Indian policy 2 has been for the Office of Historical Trust Accounting and the 3 Department of the Interior, the Department of Justice, and the 4 Department of the Treasury. 5 Q. Have you written my papers for these clients? 6 A. Yes, I have. I've written a number of major papers, 7 research papers, that have run between about 20 and 75 typed 8 pages. 9 Q. Are some of those papers reflected on the screen that's 10 shown now? 11 A. Yes, they are. And as you can see, I've written somewhat 12 about natural resource issues, but again, the primary focus has 13 been on the government's administration of Indian trust funds. 14 As you can see here, the Treasury as well in the administration 15 of travel trust funds, investment of travel trust funds. I've 16 written on IIM banking policy. And if you can scroll to the 17 next page on my resume. 18 MR. SIEMIETKOWSKI: Technical difficulties. They're 19 not getting in. Let's hold up a second, Dr. Kehoe, until we get 20 the screen working on both sides. 21 BY MR. SIEMIETKOWSKI: 22 Q. You wanted to refer to the papers that you've written? 23 A. As noted here, in addition to those research papers I've 24 also submitted 29 reservation histories to the Office of 25 Historical Trust Accounting. These are brief studies about 15 Jacqueline M. Sullivan, RPR Official Court Reporter 1071 1 to 20 pages, primarily narrative, but with some historic data 2 focused primarily on the history of land tenure on that 3 reservation, focusing on allotment, changes in land tenure over 4 time, and also the history of natural resource development on 5 those particular reservations. Of course, every reservation has 6 its own unique history. 7 Q. Have you performed any Indian research at the National 8 Archives and its facilities? 9 A. Yes. That's one of the primary places where we conduct 10 research here at the main building in Washington, D.C. and also 11 at College Park and also at some of the regional archives 12 scattered across the country. 13 Q. Have you conducted any Indian records research at the 14 American Indians Record Repository? 15 A. Yes. I've made a number of trips to Lenexa to conduct 16 research. 17 Q. What about federal records center? 18 A. I've also conduced work there as well. 19 Q. You mentioned that Interior is one of your clients. Have 20 you conducted any Indian research at Interior? 21 A. Well, I've made extensive use of the Department of Interior 22 library in their main building, and I also made several research 23 trips to what was called the Office of Trust Records in 24 Albuquerque, New Mexico, working in more recent Indian records, 25 and again I believe most of those records have been moved to the Jacqueline M. Sullivan, RPR Official Court Reporter 1072 1 facility in Lenexa. 2 Q. Dr. Kehoe, do belong to any professional organizations? 3 A. I belong to several. American Society For Environmental 4 History, the Business History Conference, and the Organization 5 for American Historians. 6 Q. Earlier, Dr. Kehoe, you mentioned teaching at Wake Forest 7 University. Have you taught any Indian courses here in 8 Washington, D.C.? 9 A. I team-taught a course with other staff members at Morgan 10 Angel in 2000 at George Washington University. 11 Q. Did that course involve Indian matters? 12 A. The primary focus was the history of federal policy towards 13 Native Americans. 14 Q. How does the Indian research you've done for Morgan Angel 15 compare with the research you did to earn your Ph.D? 16 A. Although the topics are very different, they're both 17 examples I think of public policy research that is doing 18 extensive research at the archival level and government records 19 trying to trace changes in policy over time. I would say it's 20 fair to say I've done far more research and writing for Morgan 21 Angel than I did during my Ph.D at Ohio State. 22 Q. Thank you, Dr. Kehoe. 23 At this time, your Honor, we offer Dr. Kehoe as an 24 expert in the history of federal Indian relations and in federal 25 Indian records. Jacqueline M. Sullivan, RPR Official Court Reporter 1073 1 THE COURT: Any objection or voir dire? 2 MR. HARPER: Yes, a little voir dire, if I could, your 3 Honor. Good afternoon, your Honor. 4 THE COURT: Good afternoon. 5 VOIR DIRE 6 BY MR. HARPER: 7 Q. Good afternoon, Dr. Kehoe. I'm Keith Harper, counsel for 8 the plaintiffs. I'm just going to ask you a few questions about 9 your experience. 10 A. Certainly. 11 Q. First let me ask you, are you an accountant? 12 A. No, I am not. 13 Q. Are you an econometrician? 14 A. No, I am not. 15 Q. Are you an economist? 16 A. No. 17 Q. Statistician? 18 A. No, I am not. 19 Q. Do you claim any expertise in the management practices of 20 the Office of the Comptroller of the Currency? 21 A. I'm somewhat familiar with the comptroller of the currency 22 and its historical role with regard to failed banking 23 institutions. That's been part of my research. 24 Q. Again, do you claim expertise in the management practices 25 and policies of the Office of the Comptroller of the currency, Jacqueline M. Sullivan, RPR Official Court Reporter 1074 1 do you consider yourself an expert in that? 2 A. No, I do not. 3 Q. Do you consider yourself an expert in the United States 4 budgetary processes? 5 A. No, I do not. 6 Q. Do you consider yourself an expert in the debt management 7 practices of the Department of Treasury? 8 A. No, I do not. 9 Q. Do you consider yourself an expert on the management 10 practices at the Department of Treasury? 11 A. No, I do not. 12 Q. Do you believe that you are an expert in the borrowing 13 decisions and the way borrowing decisions are made by the 14 Department of Treasury? 15 A. No, I am not. 16 Q. Do you have any expertise regarding the national debt? 17 A. No, I do not. 18 Q. Do you have any expertise in the rules and regulations 19 associated with the regulation of national banks? 20 A. No, I do not. 21 Q. Do you have expertise in what's called the FDIC? Do you 22 know the term FDIC, Federal Deposit Insurance? 23 A. Yes, I'm familiar with its general rule and how federal 24 deposit insurance is applied. 25 Q. Would you consider yourself an expert in that field? Jacqueline M. Sullivan, RPR Official Court Reporter 1075 1 A. No, I would not describe myself as an expert in that field. 2 Q. Do you consider yourself an expert in public securities? 3 A. No, I do not. 4 Q. Do you consider yourself an expert on how monies are 5 collected at the Department of Treasury? 6 A. Again, I have some familiarity with these procedures, but, 7 no, I would not describe myself as an expert in that area. 8 Q. And what about how disbursements are made from the 9 Department of Treasury? 10 A. My answer would be the same. 11 Q. Do you consider yourself an expert on what is called the 12 Treasury general account? 13 A. No, I do not. 14 Q. Now, you talked a little bit about your experience with 15 Indian Affairs? 16 A. Yes, sir. 17 Q. Do you recall that? And I noticed on your resume, 18 however -- we can look on the publication section -- are any of 19 the actual publications and journals that you have written, they 20 seem to be environmental principally; is that correct? 21 A. My publications have been in the field of environmental 22 history, that is correct. 23 Q. So you haven't had any publications, any journals presently 24 as of today on any subjects related to Indian Affairs; is that 25 fair? Jacqueline M. Sullivan, RPR Official Court Reporter 1076 1 A. No, that's right. 2 Q. So that is true, that you do not have any, correct? 3 A. That is true. 4 Q. Thank you. And what areas of Indian policy do you consider 5 yourself an expert? 6 MR. SIEMIETKOWSKI: Objection, your Honor; relevance. 7 We've offered him as an expert in two particular areas only. 8 MR. HARPER: I understood the offer was for generally 9 on Indian Affairs, your Honor. I mean, that's a -- 10 THE COURT: Stand up and repeat your labels for the 11 areas in which you expect to ask questions. 12 MR. SIEMIETKOWSKI: We asked to qualify Dr. Kehoe, 13 your Honor, as an expert in the history of federal Indian 14 relations and in federal Indian records. 15 MR. HARPER: Your Honor, federal Indian relations is a 16 very broad topic area. 17 THE COURT: I understand that. I understand that, but 18 he knows a lot more about it than I do, and we're talking about 19 aiding the finder of fact here, so if he's got something that 20 will help me I'm going to listen to it. 21 MR. HARPER: Thank you, your Honor. Well, then, at 22 this time I will withdraw my objections on those two topic 23 areas. 24 THE COURT: All right. Thank you. 25 BY MR. SIEMIETKOWSKI: Jacqueline M. Sullivan, RPR Official Court Reporter 1077 1 Q. Dr. Kehoe, let's talk first about IIM banking policy, 2 specifically where the IIM was. Have you looked at the 3 placement of IIM throughout its history? 4 A. Yes, I have, and historically I think I can say that IIM 5 has been placed in three general areas or categories, and that 6 is in commercial banks, invested in federal securities, or in 7 checking accounts with the Treasury under the control of BIA 8 disbursing agents. 9 Q. Now, as you discussed these three categories during your 10 testimony, Dr. Kehoe, have you prepared any exhibit which might 11 assist in your testimony today? 12 A. I prepared a Power Point presentation. 13 Q. I ask you to look at DX 497 on your screen, please. This 14 is a twelve-slide Power Point show. Do you recognize it? 15 A. Yes, I do. 16 Q. Who prepared it? 17 A. I prepared it. 18 Q. I ask you now, Dr. Kehoe, to look at slide two, please. 19 Dr. Kehoe, why does your slide or your timeline begin in 1898? 20 A. I chose 1898 because the law was enacted that made BIA 21 disbursing agents legally accountable under their personal bond 22 for all monies that came into their possession, the key being 23 all monies, not just public funds, and this clarified their 24 legal responsibility for handling IIM. 25 Q. Dr. Kehoe, what's the significance for the Court of the Jacqueline M. Sullivan, RPR Official Court Reporter 1078 1 three dates on slide two? 2 A. Well, these are three important milestones, I would say, in 3 the history of the handling of IIM. 4 Q. Now, regarding 1908, what kind of deposits are we talking 5 about there? 6 A. Well, this law was giving formal authority or recognition 7 to a policy BIA had been following for some time. Since at 8 least 1899 they had been placing IIM in commercial banks either 9 in checking accounts or in time deposits. 10 Q. And what's the difference between a time deposit and 11 another kind of deposit? 12 A. Well, to put it simply, in a time deposit the depositor 13 guarantees that the funds will be placed in the bank for a 14 certain amount of time and will therefore receive a certain rate 15 of interest because of that. 16 Q. And what's the contrasted? 17 A. Contrasted with a demand deposit or what they would call a 18 checking account in which the money is available at any time. 19 Q. Was interest paid on those typically? 20 A. In this period they were. 21 Q. Let me show you next slide, three, Dr. Kehoe. Now, why 22 does this period go up to 1933 in your analysis? 23 A. As I'll be discussing, 1933 is when an important federal 24 law was enacted that really had a big impact on the management 25 of the IIM. Jacqueline M. Sullivan, RPR Official Court Reporter 1079 1 Q. And what law was that? 2 A. That was the Banking Act of 1933, sometimes referred to as 3 the Glass Ceiling Act. 4 Q. Before we get to that, let's talk about the different 5 columns here on slide three. What is meant by in "bonded banks 6 to the credit of individual Indians"? 7 A. Before we go any further, Mr. Siemietkowski, can I just 8 make a comment about what I tried to achieve with these tables 9 that we will be discussing? 10 Q. Go right ahead. 11 A. I'll be providing figures for various years throughout the 12 20th century. What I'm trying to do is to show for years in 13 which complete data is available where all IIM was kept at any 14 particular time. Using those three categories, bank, securities 15 and then the Treasury, and as you'll see I only used years when 16 I felt a complete picture was available that would allow us to 17 fill in all those categories. 18 Q. So with that explanation in mind, what's meant then by "in 19 bonded banks"? 20 A. Well, this refers to the practice we just discussed of 21 placing IIM in commercial banks for individual Indians. 22 Q. And what about "in the hands of disbursing officers"? 23 A. I have not been able to determine exactly what that phrase 24 means. I think it's quite likely that refers to the IIM that 25 was kept in Treasury checking accounts. Jacqueline M. Sullivan, RPR Official Court Reporter 1080 1 Q. Now, the dollar value figures, Dr. Kehoe, where are they 2 from? 3 A. For this period they're taken from the annual reports of 4 the commissioner of Indian affairs. 5 Q. What about the percentage figures, who were they taken 6 from? 7 A. Those were just simple calculations that we did. 8 Q. Well, let me show you next DX 33 previously admitted in 9 October. Do you recognize that, Dr. Kehoe? 10 A. Yes. I believe the Court is probably quite familiar with 11 this document by now. It's the annual report of the 12 commissioner of Indian affairs for 1910. 13 Q. I ask you next to look at page two of this document, Dr. 14 Kehoe. And what if any data did you use on this for your Power 15 Point slide that you were just discussing? 16 A. In the highlighted area I drew upon two figures. The 17 figure of approximately $2.6 million for in hands of disbursing 18 officers, and the figure for approximately $6.8 million in 19 bonded banks to credit of individual Indians. 20 Q. Now, to see this more clearly I ask that we look at the 21 first split screen. Could you kind of talk us through those two 22 screens and how they transpose in terms of data from one to 23 another? 24 A. Well, I think it's pretty straightforward. In balance -- 25 excuse me, in hands of disbursing officers is under that second Jacqueline M. Sullivan, RPR Official Court Reporter 1081 1 column for 1910 that says in hands of disbursing officers, and 2 that $6.8 million figure in bonded banks to credit of individual 3 Indians is in that first column next to 1910. 4 Q. Let's go to slide four then of your slide show then, Dr. 5 Kehoe. Now, why here, Dr. Kehoe, do the top horizontal columns 6 change names? 7 A. Well, in this period in the subsequent periods that we'll 8 be discussing, the data is more straightforward, I believe, and 9 with some confidence we can place it in one of these three 10 categories, that is, in banks, in federal securities, held at 11 Treasury. 12 Q. Now, you talked before about funds in banks. What's meant 13 by funds invested in the federal securities? 14 A. That refers to IIM that was invested in Treasury bills, 15 notes and bonds, Treasury securities, you might refer to them 16 as, and later on this could also include what were called agency 17 issues. That is securities that could be issued by Fannie Mae, 18 the small business administration; agencies like that. 19 Q. What about that time period up to 1933, can you provide the 20 Court of any examples of what type of federal securities are in 21 that time period? 22 A. Well, this is before the 1938 law that expanded the range 23 of federal securities that IIM could be invested in, so this 24 would primarily consist of Treasury securities of different 25 kinds, different maturity dates and so forth. Jacqueline M. Sullivan, RPR Official Court Reporter 1082 1 Q. Let's turn next to the next slide, Dr. Kehoe. Why does the 2 next period in your analysis go to 1966? 3 A. 1966, as we'll be discussing, is when the BIA began to 4 centralize IIM investment. Prior at this it had been quite 5 decentralized. 6 Q. Let's talk, before we get to that, about 1933. You touched 7 upon this briefly a moment ago, but what was the impact as 8 reflected in your second sentence there of glass ceiling? 9 A. Well, glass ceiling prohibited banks from paying interest 10 on demand deposits or checking accounts, so the BIA had no 11 choice but to close all of the checking accounts at commercial 12 banks that had been handling IIM, and move those funds into the 13 Treasury checking accounts controlled by special disbursing 14 agents. 15 Q. And Dr. Kehoe, what's the significance of 1938? 16 A. This is an important general investment law for Indian 17 trust funds. I believe its basic structure is still in place 18 today and it lays out the legal investments for IIM, again, 19 federal securities, and also it can be placed in banks, paying a 20 more reasonable rate of interest. 21 Q. Dr. Kehoe, the Great Depression happened during this time 22 period, did it not? 23 A. Yes, sir. 24 Q. Did the Great Depression affect IIM? 25 A. It certainly did. It affected everything in the United Jacqueline M. Sullivan, RPR Official Court Reporter 1083 1 States. At the reservation level certainly this slowed leasing 2 and other economic operations, so IIM revenue would certainly be 3 decreased. Oh, and it certainly impacted the financial sector. 4 Many banks failed during this period. 5 Q. Did any of those banks that failed include IIM? 6 A. Yes. Quite a few did. 7 Q. And how, if at all, did Interior and Congress react to 8 those bank failures? 9 A. Well, the BIA had been following a policy for some time of 10 securing IIM deposits either by requiring banks to purchase a 11 surety bond or else to purchase federal securities and to post 12 them as collateral. 13 Q. How effective were those surety bonds or collateral in 14 recovering any losses? 15 A. My research indicates that those safeguards were quite 16 effective, and just about all the cases that I've seen discussed 17 or referred to it appears that all of that money that was 18 temporary unavailable because the bank closed was made available 19 to the BIA when the surety bond paid up or those collateral 20 securities were turned over to the BIA. 21 Q. Dr. Kehoe, did tribal IIM have a role during this time 22 period? 23 A. Well, I believe that the tribal IIM accounts really 24 originated in the 1930s after the Indian Reorganization Act, and 25 my research indicates that they were invested very much like the Jacqueline M. Sullivan, RPR Official Court Reporter 1084 1 other types of IIM accounts. 2 Q. Let's move from slide five and show you slide six, Dr. 3 Kehoe. What is the significance in the first bullet there? 4 A. Well, this indicates a major trend in investments. Between 5 the mid-30s and the mid-1940s almost all of the IIM that had 6 still been maintained in time deposits was gradually shifted 7 into federal securities so very little money in banks by, say, 8 the late 1940s. 9 Q. And what about 1960, why did you add that date to the 10 slide? 11 A. This is when the commissioner of Indian affairs began to 12 implement a policy in which once again large amounts of IIM 13 would be placed back in commercial banks, I should note strictly 14 for purposes of investment, that is, in time deposits, later 15 certificates of deposit, but not in checking accounts. 16 Q. I'm going to show you slide seven, Dr. Kehoe. What trend, 17 if any, is evident on this chart? 18 A. Well, I think it's clear that we see a decline in IIM kept 19 in commercial banks and federal securities become overwhelmingly 20 the main avenue of investment with the remaining funds held in 21 the Treasury. 22 Q. Let me show you DX 408, if I could briefly, Dr. Kehoe. Now 23 I'll ask you to look at page seven of this, please. What is 24 this, if you recognize it, please? 25 A. This is part of one of those annual reports of the balances Jacqueline M. Sullivan, RPR Official Court Reporter 1085 1 of Indian service special disbursing agents that the BIA 2 submitted to the Treasury on an annual basis during this period. 3 Q. Now, using our second split screen, I'll ask you to please 4 describe for the Court how you used that data for use in your 5 Power Point slide. 6 A. This requires a little more explanation. Down at the 7 bottom left-hand column of the lower excerpt deposits in banks 8 and cash and personal possession of disbursing officers, $3.9 9 million roughly that goes to the column in my table IIM system 10 funds in banks. Above that under that subheading with the 11 Treasury of the United States it reads subject to check of. 12 $6.3 million. That was placed in the column for IIM system 13 funds held at Treasury. The remaining two values, investments 14 and first we see investments with the Treasury of the United 15 States and then investments with other depositories, adding 16 those two together we get the total for IIM invested in federal 17 securities. 18 Q. Let's talk next about slide eight, Dr. Kehoe. Now, why 19 does this chart have less data? 20 A. Well, this reflects the fact that between the mid-1940s and 21 the mid-'60s I was able to find just a couple of years which 22 provided hard data that addresses all three of these general 23 categories: banks, securities, and held at Treasury. 24 Q. We'll move next to slight nine, Dr. Kehoe. Now, why are 25 these dates important? Jacqueline M. Sullivan, RPR Official Court Reporter 1086 1 A. Well, as I mentioned, 1966 is when the BIA took steps to 2 centralize the investment function, and in 1985 has been 3 referred to at the beginning of the so-called electronic era in 4 terms of IIM accounting. 5 Q. You talk a little bit about your 1980s bullet? 6 A. Well, this refers to the fact that the 1980s in a way was 7 so much similar to the 1930s. It was a period of instability in 8 the financial industry and as a result some IIMs were lost when 9 depository institutions of different kinds, banks, savings and 10 loans, credit unions, became insolvent. 11 Q. Now, you talk about funds being lost. Do you know whether 12 Interior or Congress reacted to that at all? 13 A. Well, eventually when, and we're talking here about funds 14 that were lost and could not be regained through federal deposit 15 insurance coverage or through those collateral securities, in 16 those cases the Interior department went to Congress and 17 requested that money be appropriated to cover those loses, which 18 it was -- 19 Q. I'm sorry. What was the last part of your answer? 20 A. Money was appropriated to cover those losses. 21 Q. Dr. Kehoe, let's talk next about slide ten, please. I want 22 to focus you here initially on the note on the far right column. 23 What does that refer to? 24 A. This refers to the fact that by this period, the 1970s, IIM 25 investment was completely centralized and investing as much IIM Jacqueline M. Sullivan, RPR Official Court Reporter 1087 1 as at all possible was a priority for the BIA investment branch. 2 Those wanted to keep IIM fully invested as much as they could. 3 Therefore, I made the assumption that together IIM and banks and 4 in securities constitutes the great majority of the IIM system 5 funds in any particular year, but keeping in mind that there was 6 always a small amount of IIM on any particular day that would 7 not have been invested. 8 Q. Now, what trend, if any, is evident on this chart? 9 A. Well, I think you see that during this period money could 10 shift back and forth in its concentration in terms of 11 investment, that is, in certain years the majority of it is in 12 federal securities and in other years it's in banks, 13 certificates of deposit, and then this just reflects the fact 14 that the investment branch was intent on trying to obtain the 15 highest return possible for IIM. 16 Q. Let me show you now, Dr. Kehoe, DX 380. If we can enlarge 17 that a bit so Dr. Kehoe can see the first page, please. Do you 18 recognize that, and, if so, what is it? 19 A. Yes, I do. That's one of the annual investment reports 20 that was issued by the Bureau of Indian Affairs during this 21 period. 22 Q. And I'll direct your attention to page nine of this report. 23 What does this show, Dr. Kehoe? 24 A. Well, this shows the status of IIM investments as of the 25 end of fiscal year 1980, and like the other examples we've Jacqueline M. Sullivan, RPR Official Court Reporter 1088 1 looked at, this represents a snapshot in time where IIM was 2 invested at this point in time. 3 Q. Now, once more we'll go to the third and last split screen, 4 and I'll ask you to please describe for the Court how you used 5 that data for your Power Point slide. 6 A. Well, looking at the lower excerpt, the one under the 7 September 30th, 1980 heading, if you go down to the bottom row, 8 deposit funds, ISSDA, that refers to the IIM, about 230,000 9 accounts, and if you move over to the first column you see about 10 $135,800,000 in time certificates of deposit, that is bank CDs, 11 I've taken that figure and placed in the column for IIM system 12 funds in banks. 13 Q. Thank you. Let's look next at slide, eleven, Dr. Kehoe. 14 Now, why does that first bullet cover a two-year period? 15 A. Well, again, 1986 is when the so-called electronic era 16 started in IIM accounting, and I thought it appropriate to try 17 to take the story up to the present day, 2008. 18 Q. Now, the last bullet there, 1995, is that the same 19 information you were testifying to a few moments ago regarding 20 covering losses? 21 A. That's right. Ultimately approximately $5.7 million was 22 appropriated by Congress to reimburse IIM trust funds for money 23 that had been lost when depository institutions failed, and I 24 should be clear on this. They reimbursed principal lost at 25 these failed institutions and interest that was earned up to the Jacqueline M. Sullivan, RPR Official Court Reporter 1089 1 date of the closure of these institutions. 2 Q. Lastly, in your slide show, Dr. Kehoe, we'll look at slide 3 number twelve. What are these? 4 A. This merely refers to the sources for the data in the 5 various tables that I've discussed. 6 Q. Are these sources publicly available? 7 A. Yes. Some of them could probably be obtained at libraries, 8 Didn't of Interior library, maybe the Library of Congress, and 9 the others I believe are all collected at various archives. 10 Q. Thank you. Now, when you reviewed of all these sources, 11 Dr. Kehoe, and in your preparation of these slides, didn't you 12 see any evidence of IIM sitting outside of those three 13 categories you described at the beginning of your testimony? 14 A. No, no, I have not. 15 Q. In reviewing your sources and in preparing your slide 16 presentation, did you see any evidence of IIM wrongfully 17 withheld from beneficiaries? 18 A. No. During certain periods it's clear that the BIA was 19 withholding money from accountholders, but this was considered 20 the proper policy at the time. There was nothing illegal about 21 it. That was just the policy. 22 Q. Dr. Kehoe, let's change gears here now, and having talked 23 about where the IIM has been throughout the history of the IIM 24 system, let's talk about the Plaintiffs' calculation of alleged 25 benefit to the government. Have you reviewed and analyzed Jacqueline M. Sullivan, RPR Official Court Reporter 1090 1 Plaintiffs' March 19 remedies brief, including its attachments? 2 A. Yes, I have. 3 Q. Have you reviewed and analyzed Plaintiffs' April 22 reply 4 brief? 5 A. Yes, I have. 6 Q. Do you recall the term "in bonded banks to the credit of 7 individual Indians" from your slides? 8 A. Certainly. 9 Q. Dr. Kehoe, does your historical research show any benefit 10 to the government from funds held in bonded banks to the credit 11 of individual Indians? 12 MR. HARPER: Objection, your Honor. This is well 13 beyond the scope of his expertise. 14 THE COURT: Sustained. 15 BY MR. SIEMIETKOWSKI: 16 Q. Dr. Kehoe, in looking at the historical documents which 17 you've reviewed to prepare your slide show, did those documents 18 show anything or indicate anything or ever address in any way a 19 benefit to the government from funds held in government 20 securities? 21 MR. HARPER: Objection, your Honor; relevance. 22 THE COURT: Sustained. "Relevance" isn't your 23 objection, but I'll sustain the objection anyway. 24 BY MR. SIEMIETKOWSKI: 25 Q. Dr. Kehoe, let me show you slide four of your presentation Jacqueline M. Sullivan, RPR Official Court Reporter 1091 1 if I could, please. Regarding the far right column, system 2 funds held at Treasury, do you have from your historical 3 research any familiarity with the ability of the government to 4 spend funds in that column? 5 MR. HARPER: Objection, your Honor. He testified that 6 he is not an expert with respect to anything related to the 7 Department of Treasury. 8 MR. SIEMIETKOWSKI: Your Honor, we're not asking -- 9 MR. HARPER: This is about Department of Treasury. 10 THE COURT: I'll overrule that objection and let the 11 witness answer, if he can. 12 THE WITNESS: Could you please restate the question? 13 BY MR. SIEMIETKOWSKI: 14 Q. Yes. Dr. Kehoe, do you have through your historical 15 research any familiarity with the government's ability to spend 16 the funds that you reference in the far right column held at 17 Treasury? 18 A. As I mentioned, this refers to funds that were held in what 19 were called Treasury checking accounts that were under the 20 supervision -- under the control, rather, of BIA disbursing 21 agents in the field, and they certainly had the authority to 22 disburse IIM from these checking accounts just as they did to 23 disburse IIM from banks. 24 Q. Now, Dr. Kehoe, looking at slide four, during this time 25 period, 1898 to 1933, where was most of the IIM held? Jacqueline M. Sullivan, RPR Official Court Reporter 1092 1 THE COURT: That kind of speaks for itself, doesn't 2 it, counsel? 3 MR. SIEMIETKOWSKI: It does, your Honor. What I would 4 like then to do is show the Court slide seven once the court has 5 finished looking at slide four. 6 THE COURT: I'm finished. 7 MR. SIEMIETKOWSKI: Slide seven, please. 8 And then again, once the Court is finished we'd like 9 to show slide eight. 10 THE COURT: I'm finished with that one and that one. 11 MR. SIEMIETKOWSKI: Slide ten, please. 12 THE COURT: And that one. 13 MR. SIEMIETKOWSKI: And slide eleven. 14 THE COURT: All right. 15 BY MR. SIEMIETKOWSKI: 16 Q. Dr. Kehoe, from your review of Plaintiffs' two pleadings 17 which were earlier discussed, did they ever consider where the 18 money was in arguing for an added benefit to the government? 19 A. There appeared to be an assumption that the money was 20 always kept in the U.S. Treasury, and that would appear to me to 21 be a weakness of the model of that assumption. 22 MR. HARPER: Again, your Honor, I just renew my 23 objection. Again, that is well beyond the scope of his 24 expertise. 25 THE COURT: I'm going to allow the answer, but barely. Jacqueline M. Sullivan, RPR Official Court Reporter 1093 1 MR. SIEMIETKOWSKI: Obviously, your Honor, the witness 2 is testifying as a historian. If he's testifying as to where 3 the money is historically. 4 THE COURT: Look, I gave a little brief little seminar 5 on this whole subject yesterday morning. The burden hasn't even 6 shifted to the government on this question yet in my view, and 7 you're basically now adding filigree to the numbers that you've 8 already shown me. If somebody from the Treasury department 9 wants to come in here and tell me about how those balances are 10 dealt with in the Treasury department I'll listen to it, but I 11 don't think it's really a historical matter. 12 MR. SIEMIETKOWSKI: Yes, your Honor. 13 BY MR. SIEMIETKOWSKI: 14 Q. To conclude your testimony then, Dr. Kehoe, what is your 15 opinion regarding what the historical records show regarding the 16 placement of IIM? 17 A. Well, I would say the historical record indicates that 18 throughout the 20th century the majority of IIM was kept outside 19 of the Treasury. In some years the great majority and that 20 those funds kept outside the Treasury also earned interest for 21 the accountholders. 22 MR. SIEMIETKOWSKI: Thank you, your Honor. That 23 concludes my direct. 24 We would like to move the admission of two exhibits. 25 That would be DX 497, Dr. Kehoe's Power Point show, and then DX Jacqueline M. Sullivan, RPR Official Court Reporter 1094 1 498, his C.V. as well. 2 THE COURT: 497 and 498 will be received. 3 MR. SIEMIETKOWSKI: Thank you, sir. 4 (Exhibit Nos. DX 497 and DX 498 received into evidence 5 at about 4:12 p.m.) 6 CROSS-EXAMINATION 7 BY MR. HARPER: 8 Q. Dr. Kehoe, I'd like to start with your Power Point 9 presentation, if I could. 10 A. Certainly. 11 Q. And ask that we turn to DX 497, 0010. Do you recall 12 testifying about this page? 13 A. Yes, I do. 14 Q. Now, first I'd like to ask you, who came up with this Power 15 Point and the titles in the Power Point? 16 A. This is my creation, this Power Point. 17 Q. Yours and yours solely? 18 A. I made the final decisions about what to include. 19 Q. Who else was involved in forming the analysis as to what to 20 include? 21 A. Well, I also collaborated with a few members of the staff 22 at Morgan Angel. 23 Q. And that was it? 24 A. Could you clarify your question as to what was involved in 25 creating the Power Point? Jacqueline M. Sullivan, RPR Official Court Reporter 1095 1 Q. Let me ask some specifics about it. 2 A. Okay. 3 Q. You have three columns, correct? 4 A. Yes, I do. 5 Q. And you have three columns in a number of different eras, 6 is that a fair statement? 7 A. Yes, I do. 8 Q. And the same three columns, correct? 9 A. Yes. 10 Q. And one of those three columns is IIM System Funds in 11 Banks, correct? 12 A. That's right. 13 Q. Okay. Who came up with that specific title? 14 A. The Department of Justice requested that we use the term 15 IIM system funds to be consistent with other experts. 16 Q. The Department of Justice directed you to use that title? 17 A. They suggested it, and I was fine with it because before 18 that I was using Total IIM, which I believe is synonymous with 19 IIM system funds. 20 Q. Okay. What about the notion of IIM System Funds in Banks? 21 A. I mean, I have always thought of it and originally used 22 that descriptor in banks. 23 Q. Now, let me ask you to first take note of that, of this 24 page where you say IIM System Funds in Banks, and then if you 25 could flip back to 3004 where it says IIM System Fund in Banks. Jacqueline M. Sullivan, RPR Official Court Reporter 1096 1 Now, are the funds on page 004 from the period 19 -- excuse me 2 -- 1899 to 1933 and the ones later on on 010 from 1966 to 1985, 3 when you use that term IIM System Funds in Banks, are those 4 funds treated any differently in those two periods? 5 A. I'm not sure I understand your question. Could you 6 rephrase it for me? 7 Q. Okay. So we have two periods of time where you use the 8 same diction, correct? 9 A. Yes. 10 Q. To describe how funds are being held, correct? 11 A. That's right. 12 Q. Are they being held in identical ways in both instances? 13 A. No, they are not. 14 Q. So you use the exact same term but they're held in really 15 very different ways, correct? 16 A. They're held in somewhat different ways. It's a general 17 descriptive term that I think is appropriate. 18 Q. Okay. In the later period when you're saying IIM system 19 funds held in banks, how were they held? 20 A. Primarily through the purchase of bank CDs, certificates of 21 deposit. 22 Q. By whom? 23 A. Well, they're purchased on behalf of IIM accountholders. 24 Q. By whom? 25 A. By the Bureau of Indian Affairs. Jacqueline M. Sullivan, RPR Official Court Reporter 1097 1 Q. By the Bureau of Indian Affairs or the Department of 2 Treasury? 3 A. I'm trying to recall exactly who executed the transactions 4 purchasing the CDs. If the Treasury didn't execute the 5 transactions they were still acting at the behest of the BIA. 6 It was the BIA who was making these investment decisions. 7 Q. Fair enough. BIA is making the investment decisions but 8 the actual action taken on whether to purchase a security or 9 whether to invest it in banks is one done by the Department of 10 Treasury, correct? 11 A. Could you rephrase the question again? I want to make sure 12 I understand the word correctly, Mr. Harper. 13 Q. Whether you're investing in banks or investing in federal 14 securities, the person deciding where to invest it is the 15 Department of Treasury, correct? 16 A. No. No. That's exactly wrong. The person making the 17 decision about how and where to invest it is the Bureau of 18 Indian Affairs. 19 Q. Actually it's OFTM, isn't it? 20 A. After 1989. 21 Q. So prior to 1989 the Bureau of Indian Affairs is making 22 this decision but it's executed by the Department of Treasury, 23 they're the ones taking the action on either buying the federal 24 securities or investing it in banks, correct? 25 MR. SIEMIETKOWSKI: Objection, your Honor; asked and Jacqueline M. Sullivan, RPR Official Court Reporter 1098 1 answered. 2 THE COURT: Overruled. 3 THE WITNESS: Well, I think we need to look at 4 different periods. If we're looking at this period that you 5 have on the screen, 1928 to 1933, based on the research I've 6 done, Treasury really had no involvement in placing IIM in 7 banks. 8 BY MR. HARPER: 9 Q. Let me ask you a more specific question and we can again 10 turn to DEX 4970010. If we can focus on the final line, 1985. 11 A. Very well. 12 Q. Can you make that little bigger, Antonio? Thank you. 13 Now, you have $335 million under the heading IIM 14 systems funds in banks, correct? 15 A. That is correct. 16 Q. And what this really is is funds that are invested at banks 17 for commercial banks for investment purposes, correct? 18 A. That's right. 19 Q. So that is very different from what I think you're 20 testifying in the earlier period, which is that there was no 21 involvement by the Treasury department whatsoever, correct? 22 Here there is. 23 A. In the earlier period that we're looking at, 1920s, 1930s, 24 placing IIM in banks, no involvement by the Treasury department. 25 During the 1980s -- I'm not sure you're correct about the role Jacqueline M. Sullivan, RPR Official Court Reporter 1099 1 of the Treasury department. The more I think about it I recall 2 there was an auction process set up in which equally qualified 3 banks would bid for having IIMs placed in their CDs and it was 4 conducted by the branch of investments in the Bureau of Indian 5 Affairs. I'm trying to recall Treasury involvement in that 6 process. I'm just not certain, Mr. Harper. 7 Q. Have you ever heard of the term 14X6039 account? 8 A. Yes, I have. 9 Q. Is this $335 million while invested in banks credited to 10 the 14X6039 account? 11 A. Well, these are funds that are part of that account which 12 represents IIM. 13 Q. So what is your answer to the question? Is the $335 14 million you have cited here credited here to the 14X6039 15 account? 16 A. Well, if it's invested in bank CDs it's outside of the 17 Treasury. 18 Q. So you're saying it's not credited to the 14X6039, is that 19 your testimony? 20 A. I'm not sure. I'm not sure exactly how at the Department 21 of the Treasury in 6039, how this was accounted for. 22 Q. So you really don't know when you say this is outside of 23 the bank, you don't know what the relationship is between this 24 and what is credited at the Department of Treasury, fair? 25 A. Well, what I'm testifying to in terms of my understanding Jacqueline M. Sullivan, RPR Official Court Reporter 1100 1 is how the Treasury department is tracking these funds and their 2 involvement. I know that if these funds are invested in bank 3 CDs they're not held by the Treasury, they're in these bank CDs 4 belonging to commercial banks. 5 Q. Let me try to get at this $335 million a different way. 6 A. Okay. 7 Q. How does the bank get this $335 million during this period? 8 A. The branch of investment would have used IIM funds 9 available to it and purchased a bank certificate of deposit from 10 a financial institution. 11 Q. And where do they get the money to do that? 12 A. It's under their control. 13 Q. But where, okay? Where is the money derived from by which 14 they are able to invest $335 million in banks? 15 A. I understand. In other words, where is the money before 16 they buy the CD? 17 Q. Precisely. 18 A. It would be in the Treasury. 19 Q. Okay. So you have Treasury funds and they come in through 20 normal deposit streams just like other Treasury funds into the 21 Department of Treasury, correct? 22 MR. SIEMIETKOWSKI: Your Honor, objection to the 23 mischaracterization of the testimony. I don't believe the 24 witness testified they were Treasury funds. I believe he 25 testified -- Jacqueline M. Sullivan, RPR Official Court Reporter 1101 1 THE COURT: Overruled. He can answer if he can. 2 BY MR. HARPER: 3 Q. Let me rephrase the question. These funds are funds that 4 belong to individual Indians, correct? 5 A. That's right. That's why they're classified as deposit 6 funds. 7 Q. But the title of the funds are held by the United States 8 government, correct? 9 A. What do you mean, the title of the funds? 10 Q. You never heard of that term, title of the funds? 11 MR. SIEMIETKOWSKI: Objection, your Honor. Counsel 12 had tried earlier to establish the lack of knowledge of the 13 witness in this particular field. Now he's pursuing a line of 14 questioning in the same field. 15 THE COURT: He's opening a door. You can drive right 16 through it on redirect if you think it's appropriate. 17 THE WITNESS: Can you explain what you mean by the 18 term "title of funds"? 19 BY MR. HARPER: 20 Q. Well, let me ask, do you know, do you not know what that 21 term means? 22 A. Title of funds? 23 Q. Who holds the title for these funds? 24 A. No, I'm not familiar with that term. 25 Q. These funds come into the Department of Treasury through Jacqueline M. Sullivan, RPR Official Court Reporter 1102 1 various commercial banks that are where deposits are made and 2 they are then credited to Treasury accounts, correct? 3 A. They would be credited to the 6039 account at the Treasury. 4 Q. Well, first they would just come in to the TGA, correct, 5 like all funds collected and deposited, they would come in first 6 to the TGA and then credited? 7 A. I'm not familiar enough with the TGA to speak to that, Mr. 8 Harper. 9 Q. So you wouldn't know one way or the other whether or not 10 when funds initially come in to the Department of Treasury that 11 are ultimately, as you put it, in funds in banks, that they 12 first go to the TGA account, you wouldn't know that one way or 13 the other? 14 A. I'm not familiar enough with the workings of the TGA 15 account to comment on that. 16 Q. And you wouldn't know that if monies were somehow not 17 properly credited to 14X6039 once after the TGA you would have 18 no idea whether those monies are either still in the TGA or 19 credited some other place in Treasury, correct, you just don't 20 have the familiarity with that? 21 A. As I testified, I'm not that familiar with the Treasury 22 accounts system once funds have entered the Treasury. Speaking 23 in terms of the broad federal central accounting system, I am 24 not. 25 Q. Do you know what the term "deposit fund" is? Jacqueline M. Sullivan, RPR Official Court Reporter 1103 1 A. Yes, I am familiar with that term. 2 Q. And what is the deposit fund? 3 A. A deposit fund refers to monies that are held by the 4 federal government for individuals or institutions outside the 5 government. In other words, it is not government funds. It's 6 restricted on what they could do with the money because it is a 7 deposit fund. 8 Q. And are those funds that are in deposit funds, are think 9 considered, quote/unquote, available to the Treasury, end quote? 10 A. I'm not sure what is meant by that phrase. 11 Q. So you've not heard the phrase "available to the Treasury"? 12 A. I've heard the phrase before. I'm not sure if I understand 13 exactly its meaning in this context. 14 Q. But what it's fair to say is that the $335 million here as 15 is under the head IIM system funds in banks went through 16 necessarily the Treasury department first? 17 A. Well, I think it's fair to say that before the money could 18 be invested it first had to be selected by the Bureau of Indian 19 Affairs in some manner and would have been deposited to the 6039 20 account and it then becomes available for investment. 21 Q. At the Department of Treasury? 22 A. It becomes available for investment by the BIA. 23 Q. But you understand that the 14X6039 is an account held at 24 the Department of Treasury, you do understand that? 25 A. I understand that. I think it's a question of who controls Jacqueline M. Sullivan, RPR Official Court Reporter 1104 1 that money and makes the investment decisions, which was the 2 BIA. 3 Q. Okay. Making the decision is one thing but who holds the 4 money is a different thing as you just testified, fair 5 statement? 6 A. Fair statement. 7 Q. Okay. So it's fair to say that when you use a term like 8 these headings here, they may mean different things in different 9 periods, fair statement? 10 A. Yes, it's a general category, and as we've just discussed, 11 there are some differences between the different periods, yes. 12 Q. Some important differences, no? 13 A. I suppose it depends on how you understand the term 14 "important." I mean, that's a judgment call. 15 Q. Do you know, Dr. Kehoe, what is meant by the term "Treasury 16 depository"? 17 A. Yes, I'm familiar with that term. 18 Q. These funds that are invested in commercial banks, when 19 they first enter the Department of Treasury do they enter 20 through Treasury depositories? 21 A. Well, if they're being deposited to the Treasury I believe 22 they would have to be deposited through a Treasury depository, 23 an official government depository. 24 Q. Can we bring up PX 136 on the front cover, please? 25 Dr. Kehoe, do you recognize this document? Jacqueline M. Sullivan, RPR Official Court Reporter 1105 1 A. I don't believe I've seen it before. 2 Q. And do you know approximately how many Treasury 3 depositories there are presently? 4 A. No, I do not. 5 Q. Would it surprise you that there were about five hundred 6 back at this date, 1985? 7 A. No, I wouldn't say that surprises me. 8 Q. For the record, this is PX 136, and it's a General 9 Accounting Office document, Report to the Secretary of the 10 Treasury entitled Improvements Needed in Controlling and 11 Accounting for Treasury Banking Arrangements. If I could turn 12 your attention to page twelve of this document. 13 MR. SIEMIETKOWSKI: Objection, your Honor; foundation. 14 The witness has said he's not familiar with the document and yet 15 counsel is asking the witness to testify regarding the document. 16 THE COURT: Well, let's see what the question is. 17 BY MR. HARPER: 18 Q. If you can just read that to yourself, Dr. Kehoe, and let 19 me know when you are ready. 20 A. I have read it. 21 Q. So basically we have FRBs and we have supplementing those 22 all these commercial banks that accept deposits on behalf of the 23 Treasury department, correct? I mean, that is how you read 24 that? 25 A. Yes. Jacqueline M. Sullivan, RPR Official Court Reporter 1106 1 Q. And those would be the places that you just identified were 2 individual Indian monies would go into the Treasury department, 3 fair statement? 4 A. I believe that's how the system works. 5 Q. And so these funds that you're saying this latter period 6 until 1985 on DX 497, 0010, 1967 to 1985, they would first enter 7 Treasury through this system and then they would ultimately get 8 invested in various commercial banks, fair statement? 9 A. I think that's accurate. 10 Q. Now, are you familiar with the practice by which banking 11 services on behalf of the United States are compensated for by 12 investing monies that do not earn interest, are you familiar 13 with that practice? 14 A. No, I am not. 15 Q. So if I could turn to page four of this same document. And 16 we can go down to the seventh bullet point and just the first 17 two sentences. Yes. So here it says, I'll just read the two 18 sentences into the record: FMS pays for banking services with 19 compensating balances, i.e. -- it doesn't say "i.e." -- non- 20 interest bearing deposits at the bank. These balances are not a 21 part of the FMS appropriation and FMS does not include them in 22 its budget. You can continue to read for context. 23 MR. SIEMIETKOWSKI: Your Honor, we have an objection 24 as to relevance. This is not tied, I don't believe, to IIM. 25 THE COURT: I'm going to allow it. Jacqueline M. Sullivan, RPR Official Court Reporter 1107 1 BY MR. HARPER: 2 Q. Did you have any knowledge prior to reading this of this 3 practice? 4 A. No, I did not. 5 Q. One other question. If we can turn yet again back to the 6 DX-49710. Concentrating once again, Dr. Kehoe, your attention 7 to this page, do you understand how or do you have any knowledge 8 as to how funds from here would then get disbursed to a 9 beneficiary? 10 A. Well, this is an investment so the investment would have to 11 be cashed in, you might say. In other words, the certificate of 12 deposit is allowed to expire and the bank pays the money and 13 interest to the BIA and then if a disbursement is appropriate 14 one is made or many are made. 15 Q. Once you get back the returns from your investment, where 16 are the returns placed? 17 A. In the Treasury, I believe. 18 Q. Do you know which account at the Department of Treasury? 19 A. I assume it would be 6039. 20 Q. But you don't know one way or the other? 21 A. Based on my knowledge of the Bureau of Indian Affairs, that 22 would seem to be logical that we go to 63039. I mean, that is 23 the account for IIM. 24 Q. And once the check is made out to an individual beneficiary 25 then where do the funds go? Jacqueline M. Sullivan, RPR Official Court Reporter 1108 1 A. It goes to the Indian when it's disbursed. 2 Q. A check is sent and it goes to the Indian once he receives 3 and cashes it, correct? 4 A. Well, I believe it's the Treasury's bookkeeping practice 5 that once a check is cut and leaves their system the 6 disbursement is considered complete, unlike a bank where the 7 disbursement is not considered complete and credited until that 8 check is presented for payment. 9 Q. But once the check is issued and sent out to the 10 beneficiary if there is a lapse in time where are the funds 11 held, if you know? If you don't know just say you don't know. 12 MR. SIEMIETKOWSKI: I'm going to object. This is 13 beyond the scope of my direct even with regard to the questions 14 I had asked regarding Treasury. 15 MR. HARPER: Your Honor, you allowed at least two -- 16 THE COURT: It's cross. Go ahead. 17 THE WITNESS: Refresh my memory, Mr. Harper. Would 18 you mind rephrasing that question? Just state the same question 19 again. 20 BY MR. HARPER: 21 Q. You testified that these securities get redeemed? 22 A. Okay. 23 Q. They or the investment in the banks get redeemed. That 24 goes into, in your view, 14X63039. Once a check is cut and it 25 is sent out where do the funds go at that point? Jacqueline M. Sullivan, RPR Official Court Reporter 1109 1 A. As I stated, my understanding is that the Treasury 2 department has treated check disbursements as leaving their 3 system once the check is cut. Therefore, from an accounting 4 standpoint, I would have to say that the money has then left the 5 IIM system once that check is cut. 6 Q. So for a six-month period it's where would you find those 7 funds where is it, where are those funds credited, they're just 8 not credited anywhere? 9 A. Well, they're considered to be in the hands of the Indian 10 to whom the check was sent. 11 Q. Well, we'll talk a little bit about that further later and 12 I'll show you some testimony and see if you agree with that. 13 Now I want to talk a little bit further about your 14 exhibit here, DX 4970010. And you had testified regarding -- 15 actually, let's turn back to page four of this exhibit. Dr. 16 Kehoe, is that all the information you were able to uncover from 17 various places, is all the information you were able to uncover 18 in this chart? 19 A. This is all the data that I was able to locate which I 20 believe provides a complete picture of where IIM is at in a 21 particular year. 22 Q. So this would not include then, is it fair to say, 23 information where you had one column but not the others? 24 A. That's right. I did not include years for which we had a 25 partial picture. Jacqueline M. Sullivan, RPR Official Court Reporter 1110 1 Q. Okay. And I'd like to bring up PX 124. Dr. Angel, do you 2 recognize this report? 3 A. You mean -- 4 Q. I'm sorry. Dr. Kehoe. I saw "Angel" here and I misspoke. 5 Dr. Kehoe, do you recognize this report? 6 A. Yes, I do. 7 Q. And you're the drafter of this document? 8 A. Yes, I am. 9 Q. And would it be fair to say that one of the things this 10 document tries to do is talk about the investment history of IIM 11 funds? 12 A. That's right, with the focus on banks, less so on 13 securities. 14 Q. And it also talks a little bit about how at various stages 15 there were people who were trying to get the highest return for 16 investment, is that a fair statement? 17 A. Yes, it is. 18 Q. And that goes all the way back to these funds being kind of 19 regularly invested in the early 1900s? 20 A. I would say that's a true statement. 21 Q. And even before that there were investments made? 22 A. You say "before that." How far back are you referring to? 23 Q. How far back would you say that they went? 24 A. Well, I have seen documents discussing depositing IIM in 25 banks as early as 1899. I could not speak to twenty years Jacqueline M. Sullivan, RPR Official Court Reporter 1111 1 before that. 2 Q. So you wouldn't know before 1899, but since 1899 there's 3 been regular investment, is that fair? 4 A. Yes, that's fair to say. 5 Q. Now I want to turn your attention to page fifteen of this 6 document. I'm sorry, it's page sixteen. I'm sorry, page 7 seventeen. 8 A. Mr. Harper, I get the sense that there might be a lot of 9 questions on this report. If that's the case, would I be able 10 to get a paper copy to use? 11 Q. Let's see how it goes and then if you like -- I only have 12 my one marked-up copy with me. I don't think you're going to 13 have as many questions on this as you may think. 14 A. Okay. 15 Q. And if it turns out differently, then we'll be glad to 16 provide one. 17 If you can turn your attention down to the bottom of 18 the page, and you note here that by 1926, Commissioner Merit 19 reports $27 million in monies held in trust for individual 20 Indians are invested at the present time in United States bonds 21 and notes. 22 A. Yes, I see that. 23 Q. And now you did not include this information in your chart, 24 though, correct? 25 A. No, I did not. Jacqueline M. Sullivan, RPR Official Court Reporter 1112 1 Q. And what was the reason for that? 2 A. Well, for one thing -- can I see the footnote to that 3 quotation or that statement that you showed me? 4 Q. Sure. 5 A. Those are figures in a letter, so it's not the end of a 6 fiscal year as my other data is. That would be one reason for 7 not including it. All of the other data is for the end of the 8 fiscal year. That figure he gave for securities looks like this 9 is from a January 23 letter. 10 Q. Okay. So all your figures and all your charts are 11 end-of-fiscal-year figures? 12 A. I believe. 13 Q. But this is a snapshot figure and those are snapshot 14 figures, correct? 15 A. That's true, but I think you want to have a snapshot of the 16 same period the exact same time. 17 Q. Because you're using different months with different -- 18 A. Different time periods you might not get an accurate 19 picture. 20 Q. But this is really a balance, correct? 21 A. That shows at that point in time how much is in securities, 22 if I'm reading this correctly. 23 Q. Okay. Let me ask you a couple questions about that notion 24 of a snapshot, and that does not, the values there associated 25 with those securities, are those face value of the securities? Jacqueline M. Sullivan, RPR Official Court Reporter 1113 1 A. I don't believe he specifies how they were valued. 2 Q. You don't know if it's the value at the purchase or at the 3 redemption? 4 A. No. You cannot tell from that statement. 5 Q. What about with respect to your numbers that you use in 6 your chart? 7 A. These were the numbers taken from government documents, and 8 I don't recall any descriptive term explaining exactly how the 9 securities were valued. 10 Q. But you understood there was a significant difference 11 between the purchase price of the government's security and what 12 you would get -- 13 A. It could be the actual market value might be quite 14 different from the price paid for the security. 15 Q. So you don't know whether these are the purchase price or 16 the redemption value? 17 A. I could not say. 18 Q. Okay. Now, if we can turn back to your Power Point, DX 497 19 and 004 again, and now I want to concentrate a little bit on the 20 middle line there. You have there a number of IIM system funds, 21 47.41 percent, dollar value $35 million, and this is for the 22 year 1928. Do you see that? 23 A. That's correct. 24 Q. Okay. These securities are held principally by the 25 Treasury department, fair statement? Jacqueline M. Sullivan, RPR Official Court Reporter 1114 1 A. It was procedure at the time for the BIA to send the stock 2 certificates -- you'll get an actual certificate of some kind -- 3 to the treasurer of the United States to hold for safekeeping. 4 Q. So it was the policy of the United States to have these at 5 the Department of Treasury? 6 A. Yes. 7 Q. If we can bring up PX 135. 8 Dr. Kehoe, do you recognize this document? 9 A. Yes, I do. 10 Q. And could you explain what this document is? 11 A. This was a report from the comptroller general, that is the 12 General Accounting Office, on its examination of Indian trust 13 funds. 14 Q. And you've relied on this report significantly, is that a 15 fair statement, in your reporting? 16 A. It is a standard source and I have used it, that is 17 correct. 18 Q. If I could turn your attention to page 84 of this document. 19 Now, as you'll recall the IIM system funds invested in federal 20 securities on your DX 497004 had a dollar figure of $35,400,177, 21 and was that taken from this report? 22 A. It is. Not this page, but it's from the report. 23 Q. Okay. And as you scroll down the page to the highlighted 24 section on the bottom, again, that's the reference where they 25 make clear that the vast majority of those funds, some 32 Jacqueline M. Sullivan, RPR Official Court Reporter 1115 1 million, was found in the custody of the Treasury department? 2 A. That is correct. 3 Q. And indeed isn't it true that it was considered not proper 4 internal controls to hold these securities by superintendents? 5 A. Generally they were supposed to be forwarded to Washington 6 for safekeeping by the treasurer. The Osage agency might be one 7 exception. They were permitted by a couple of laws in the 1920s 8 to invest IIM outside of federal securities, so if money was 9 invested it stayed in Oklahoma state bond in a real estate 10 mortgage those securities might be held at Osage in order to be 11 considered acceptable. 12 Q. And this would be true for all other periods on your line 13 there, that most -- excuse me -- not the funds, but most of the 14 investments are basically held by the Treasury department, is 15 that a fair statement? 16 A. At least through the end of the 1950s. I've collected a 17 number of audit reports from the Treasury of the United States 18 to the BIA reporting on their holdings of federal securities 19 that the BIA could check it against its own records. 20 Q. By the way, has this report that you did -- I meant to ask 21 you this when I had it up and I forgot. What we identified as 22 PX 124 -- we don't have to put it up, but it's the IIM policy 23 report that's been updated and finalized? 24 A. I have updated the report, yes. 25 Q. And what year have you brought it up-to-date to? Jacqueline M. Sullivan, RPR Official Court Reporter 1116 1 A. In the revised report I attempted to bring the narrative up 2 to the present day, that is, 2008. 3 Q. Now I'd like to go back to some of the questions we had 4 before about the practices of how money moves, IIM money 5 specifically, within the Department of Treasury. Are you 6 familiar with a gentleman by the name of -- 7 MR. SIEMIETKOWSKI: Objection, your Honor; beyond the 8 scope of my direct and of this witness' knowledge. 9 THE COURT: Sustained as to the scope of the direct. 10 BY MR. HARPER: 11 Q. Dr. Kehoe, when funds come in to the IIM system, we had 12 talked a little bit earlier about how they come in to the 13 Department of Treasury. Do you recall that discussion? 14 A. Yes, I do. 15 Q. Okay. And I'd like to bring up some testimony of 16 Commissioner Greg from the Department of Treasury. 17 MR. SIEMIETKOWSKI: Same objection, your Honor. 18 MR. HARPER: Your Honor? 19 THE COURT: I'll hear you, Mr. Harper. Where are you 20 going with this? I mean, you're talking about movement of 21 money. He's disclaimed any expertise in that. 22 MR. HARPER: Well, your Honor, I think that even 23 though there weren't specific questions on it, during the direct 24 they went over it and what was moved into evidence was an 25 exhibit which contained some information, and I think it is Jacqueline M. Sullivan, RPR Official Court Reporter 1117 1 intended to give the idea that when funds are held in commercial 2 banks there is no way for the government to get benefit from 3 those funds, but I think the answer to that is that the way 4 money goes in and out of the Department of Treasury that ends up 5 in that $335 million in 1985 held by banks is the way in which 6 you can have funds going to other accounts that aren't credited 7 there, so I want to ask this witness if he's familiar with how 8 those things go in and out, and it answers directly his exhibit. 9 THE COURT: Go ahead and ask the question, but here's 10 what I understand to be the record on this so far. The 11 plaintiffs' theory of benefit to the government is that funds on 12 deposit with the Treasury eliminate the need for Treasury to 13 borrow those monies at the ten-year T bill rate or whatever that 14 rate was. That's the theory of the plaintiffs', benefit to the 15 government case. Now, the question I suppose is whether monies 16 held by the Treasury in banks fulfills the same function, and 17 I'm not sure that this witness or any witness I've heard yet 18 answers that question. If money is being held in banks drawing 19 interest it's an interesting concept that the government can 20 sort of double-count the money that is drawing interest and be 21 able to borrow money on it. That's a pretty good trick. That's 22 the level of my interest in this subject. If this witness knows 23 anything about these subjects, go ahead, feel free. Have a 24 field day. 25 MR. HARPER: Well, your Honor, I have a couple Jacqueline M. Sullivan, RPR Official Court Reporter 1118 1 questions. If they're not helpful to the Court I'll move on. 2 BY MR. HARPER: 3 Q. If can I turn your attention to July 7, 1999, Commissioner 4 Greg's testimony, if we can. 5 MR. SIEMIETKOWSKI: For the record, your Honor, we'd 6 like to add a hearsay objection as well. 7 THE COURT: Your objection is noted in perpetuity. 8 Go ahead. 9 BY MR. HARPER: 10 Q. This is an in-court statement, your Honor. If we can turn 11 to 3350 starting at the top and through line 11, and you can 12 just let me know when you've had a chance to read that. 13 A. I've read it. 14 Q. Is that consistent with your understanding? 15 A. I'm not sure that I completely understand Mr. Greg's 16 discussion of the money flow here. 17 Q. Okay, so you would agree that he says Treasury general 18 accounts, the money flows into Treasury general accounts and 19 then into the OTFM account, and that would be what we talked 20 about earlier, 14X6039 to the extent that it was credited 21 properly, correct? 22 A. Okay. 23 Q. So you understand that part. What part do you not 24 understand of that or have a question about? 25 A. When he's talking about, quote, what we call the Treasury Jacqueline M. Sullivan, RPR Official Court Reporter 1119 1 general accounts and commercial banks, end quote, I'm not sure 2 exactly what that means. 3 Q. And I'll move on. 4 A. If I can bring up -- 5 Q. Before I bring up the exhibit, what does the term "public 6 monies" mean to you? 7 A. I understand public monies to mean government money, money 8 that is owned in effect by the government. They can do with it 9 as they please. It is their money. It's public monies. 10 Q. And you understand individual Indians trust funds for 11 purposes of rules related to how money is used were generally 12 considered public monies, correct? 13 A. No, that's not my understanding. There was a series of 14 exchanges early in the 20th century on this very issue and the 15 BIA wanted IIM to be considered public money but the Treasury 16 department was pretty firm that it should not be classified as 17 such. 18 Q. If I could put bring up PX 134. 19 And this is the best copy we have, your Honor. 20 It's an, as you can see from the top, it's a 21 comptroller general of the United States report. If we can go 22 up to the top part with the date. This looks like 1928. Dr. 23 Kehoe, do you recognize this document? 24 A. Not at this point. Maybe as we read through it it might 25 become familiar to me. Jacqueline M. Sullivan, RPR Official Court Reporter 1120 1 Q. Starting off by quoting on the first highlighted paragraph, 2 a law called section of two, I believe, if I'm reading that 3 right, after you read this could you tell me whether you 4 recognize that law? 5 A. I've read the paragraph and I cannot recall seeing this 6 before. 7 Q. Okay. If you look down at the second highlighted 8 paragraph, and I'll read that into the record. The question has 9 arisen whether or not the term "public money" as outlined in the 10 above statute apply to funds of Indian pupils and other 11 individual Indian funds held in trust by disbursing officers of 12 the Indian service. While on this particular case the deposit 13 is held in the official credit of Mr. Brown and the Treasury. 14 These funds were carried in large measure in bonded depositories 15 scattered over the country, which are not subject to the 16 jurisdiction of the Treasury department. Do you see that? 17 A. May I have a moment? 18 Q. Yes, sure. 19 A. Well, he is referring to the above statute. I think this 20 might be a case, Mr. Harper, where I'd really like to see the 21 entire document to understand just how the phrase is being used. 22 Q. Sure. 23 MR. HARPER: Your Honor, could I get a copy of it? 24 THE COURT: It's five o'clock, Mr. Harper. How much 25 more do you have for this witness? Jacqueline M. Sullivan, RPR Official Court Reporter 1121 1 MR. HARPER: I think I have about an hour more, your 2 Honor. 3 THE COURT: In that case we're not going to get an 4 answer to that question today. You can give him a copy of the 5 document. 6 MR. HARPER: I will do so. 7 THE COURT: That will be his homework assignment. 8 We'll reconvene tomorrow morning. 9 MR. SIEMIETKOWSKI: Your Honor, we would like to 10 request copies of and full copies of all the documents, exhibits 11 which plaintiffs are showing this witness. 12 THE COURT: Have shown or are showing? 13 MR. SIEMIETKOWSKI: Have shown and will show. 14 THE COURT: You know if he's on cross-examination 15 you're not going to prep him while he's on cross. You don't do 16 that. If he's on cross he's on cross. You're not to talk to 17 these people during cross-examination. 18 MR. SIEMIETKOWSKI: Your Honor, it's not so much to 19 prep the witness. It's rather for my own preparation. 20 THE COURT: You're a quick study. They'll show you 21 the documents when they show them to you. 22 MR. HARPER: Your Honor, I'll provide documents as I 23 go along. 24 THE COURT: Mr. Kirschman, I haven't seen you or heard 25 you for days. Jacqueline M. Sullivan, RPR Official Court Reporter 1122 1 MR. KIRSCHMAN: So as not to disappoint you, your 2 Honor, I'd like to raise a procedural matter, and it goes back 3 to the argument yesterday and something you said during the 4 course of the recent testimony. Our next two witnesses are 5 intended to be Treasury employees who would address the issue of 6 benefit. Your Honor just made the statement related to Dr. 7 Kehoe's testimony that -- 8 THE COURT: No, no, no. I want to hear them. 9 MR. KIRSCHMAN: That was my question, your Honor. 10 THE COURT: I want to hear them. 11 MR. KIRSCHMAN: They will be here tomorrow. 12 THE COURT: Okay. 13 All right. We'll adjourn until 9:30 tomorrow morning. 14 Thank you. 15 COURTROOM DEPUTY: This Honorable Court is adjourned. 16 (Proceedings adjourned at about 5:03 p.m.) 17 - - - 18 19 20 21 22 23 24 25 Jacqueline M. Sullivan, RPR Official Court Reporter 1123 1 I N D E X 2 3 4 WITNESSES: 5 6 FRITZ SCHEUREN 7 Cross-examination by Mr. Dorris 1004 Redirect examination by Mr. Warshawsky 1061 8 TERENCE KEHOE 9 Direct examination by Mr. Siemietkowski 1068 10 Cross-examination by Mr. Harper 1094 11 12 13 14 E X H I B I T S 15 Defendants' 16 Exhibit No. Identification Marked Admitted 17 DX 497 1094 18 DX 498 1094 19 20 21 22 23 24 25 Jacqueline M. Sullivan, RPR Official Court Reporter 1124 1 CERTIFICATE 2 I, JACQUELINE M. SULLIVAN, Official Court Reporter, 3 certify that the foregoing pages are a correct transcript from 4 the record of proceedings in the above-entitled matter. 5 __________________ JACQUELINE M. SULLIVAN 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Jacqueline M. Sullivan, RPR Official Court Reporter