1404 1 UNITED STATES DISTRICT COURT FOR THE DISTRICT OF COLUMBIA 2 3 ELOUISE PEPION COBELL, : Civil Action 96-1285 et al. : 4 Plaintiffs : : Washington, D.C. 5 v. : Monday, October 22, 2007 : 6 DIRK KEMPTHORNE, Secretary : of the Interior, et al. : 7 : Defendants : AFTERNOON SESSION 8 TRANSCRIPT OF EVIDENTIARY HEARING 9 DAY 7 BEFORE THE HONORABLE JAMES ROBERTSON 10 UNITED STATES DISTRICT JUDGE 11 APPEARANCES: 12 For the Plaintiffs: DENNIS GINGOLD, ESQUIRE LAW OFFICES OF DENNIS GINGOLD 13 607 14th Street, NW Ninth Floor 14 Washington, DC 20005 (202) 824-1448 15 ELLIOTT H. LEVITAS, ESQUIRE 16 WILLIAM E. DORRIS, ESQUIRE KILPATRICK STOCKTON, L.L.P. 17 1100 Peachtree Street Suite 2800 18 Atlanta, Georgia 30309-4530 (404) 815-6450 19 KEITH HARPER, ESQUIRE 20 JUSTIN GUILDER, ESQUIRE KILPATRICK STOCKTON, L.L.P. 21 607 14th Street, N.W., Suite 900 Washington, D.C. 20005 22 (202) 585-0053 23 DAVID C. SMITH, ESQUIRE DANIEL R. TAYLOR, JR. ESQUIRE 24 KILPATRICK STOCKTON, L.L.P. 1001 West Fourth Street 25 Winston-Salem, North Carolina 27101 (336) 607-7392 26 27 Bryan A. Wayne, RPR, CRR Official Court Reporter 1405 1 For the Defendants: ROBERT E. KIRSCHMAN, JR., ESQUIRE JOHN WARSHAWSKY, ESQUIRE 2 MICHAEL QUINN, ESQUIRE J. CHRISTOPHER KOHN, ESQUIRE 3 JOHN KRESSE, ESQUIRE U.S. Department of Justice 4 1100 L Street, N.W. Washington, D.C. 20005 5 (202) 307-0010 6 JOHN STEMPLEWICZ, ESQUIRE Senior Trial Attorney 7 U.S. Department of Justice Commercial Litigation Branch 8 Civil Division Ben Franklin Station 9 P.O. Box 975 Washington, D.C. 20044 10 (202) 307-1104 11 Court Reporter: Bryan A. Wayne, RPR, CRR Official Court Reporter 12 U.S. Courthouse, Room 6714 333 Constitution Avenue, NW 13 Washington, D.C. 20001 (202) 354-3186 14 15 16 17 18 19 20 Proceedings reported by machine shorthand, transcript produced 21 by computer-aided transcription. 22 23 24 Bryan A. Wayne, RPR, CRR 25 Official Court Reporter 1406 1 P R O C E E D I N G S 2 THE COURT: All right, sir. 3 MR. GINGOLD: Good afternoon, Your Honor. 4 THE COURT: Good afternoon. 5 (The witness resumed the stand.) 6 DIRECT EXAMINATION (Continued) 7 BY MR. GINGOLD: 8 Q. Good afternoon, Mr. Duncan. 9 A. Good afternoon. 10 Q. Mr. Duncan, this morning you testified with respect to two 11 of the problems you identified at the outset of your testimony: 12 The missing data problem and the sample selection problem; is 13 that correct? 14 A. Correct. 15 Q. There are three other problems which you identified which 16 you characterize as major. Is that a fair statement? 17 A. Yes. 18 Q. The next one is irrelevant sample design, correct? 19 A. Correct. 20 Q. What do you mean by an irrelevant sample design? 21 A. Here the notion that is captured in that concept is what 22 was the sample designed, what question was the sample designed 23 to answer. And what we're able to observe from looking at the 24 administrative record, and specifically the NORC reports in the 25 administrative record, is that the sample design was really -- 1407 1 and the LSA, the sample that was drawn, was, as its name 2 implies, really a litigation support activity or an accounting, 3 as I believe this -- litigation support accounting. 4 But it was really designed to estimate the dollar exposure 5 consistent with the kinds of things that I would do in a damage 6 case, in preparation for a mediation or a settlement discussion, 7 trying to estimate the overall exposure. And it wasn't really 8 designed to answer specifically information about the accuracy 9 and completeness of the individual transaction listing and 10 account balances, and the information that was provided inside 11 of the LSA reports really went to those gross dollar questions, 12 not corrections that would potentially apply to individual 13 accounts. 14 Q. Now, you're making this statement based on your experience 15 with regard to damages issues, aren't you? 16 A. Yes, that's correct. 17 Q. And you've done that in the course of your career; you've 18 provided expertise in that regard, correct? 19 A. In hundreds and hundreds of cases. 20 Q. So what you've seen in the NORC sample is not materially 21 different from what you yourself have done in trying to estimate 22 liability, correct? 23 A. That's correct. When that's the question on the table, 24 this would be an appropriate activity to do. 25 Q. And that's a sound approach if you're a defendant, isn't 1408 1 it, in a case, correct? If you were defending yourself? 2 A. To try to come up with some estimation, it's helpful in 3 trying to negotiate your way into, is there going to be a 4 mediation, it's helpful in trying to develop your trial 5 strategy, to have some idea of the rough magnitude. 6 Q. And that is not your understanding, however, as to whether 7 or not it is an accounting that discharges a fiduciary duty, 8 correct? 9 A. It would be inconsistent with that notion in my mind. 10 Q. Now, could you give me some examples of transactions that 11 you're aware of where accounts are misstated? Do you have any 12 idea of any, or do you have any in mind? 13 A. Of specific accounts that are misstated? 14 Q. Yes. With respect to overpayment or underpayment. 15 A. Well, the only specific information we've been provided as 16 part of the administrative record was for the Eastern Region. 17 Q. And the Eastern Region tells you what based on your 18 recollection? 19 A. Well, there are specific instances in the Eastern Region 20 analytical work done by Deloitte & Touche that show there are 21 dollar discrepancies, there are dollar differences between the 22 amount that is in the IRMS, in the electronic ledger era that 23 was sampled, and when they were reconciled back to whatever they 24 were reconciled against. There were dollar differences noted. 25 Q. Let's talk about the historical accounting project itself. 1409 1 I'll call up for purposes of our discussion Bates 33-3-14. 2 MR. GINGOLD: Your Honor, the historical -- 3 THE COURT: Bates 33-3-14? 4 MR. GINGOLD: That's correct. 5 THE COURT: Okay. 6 MR. GINGOLD: That's part 2 of what is the historical 7 accounting project, and it has -- the historical accounting 8 project has two basic Bates stamp numbers 33-1, 33-2, and 33-3. 9 Part 2 is 33-3, and the page that I'd like to discuss with 10 Mr. Duncan is 33-3-14. 11 BY MR. GINGOLD: 12 Q. Mr. Duncan, you've read this before, haven't you? 13 A. Yes, sir, I have. 14 Q. That's right. And when we're dealing -- here we're dealing 15 with sample debit transactions and sample credit transactions, 16 correct? 17 A. Correct. 18 Q. And under sample debit transactions, the statement is 19 "There were no differences observed for debit transactions," and 20 then it goes on, "therefore, there were no underpayments or 21 overpayments." 22 What is your understanding of those two bullet points? 23 A. This gets interwoven with a lot of the topics we've talked 24 about this morning, which is first of all what is an error. So 25 what they're saying here is as defined by the ASM, they did not 1410 1 identify, of the transactions that they reconciled, using the 2 different levels of reconciliation, of those that they 3 reconciled were there any differences that were observed. 4 And the other overlay here is it's not just any difference; 5 it has to be above their de minimis criteria. So there had to 6 be a difference in the transaction of at least a dollar. So 7 what they're saying here is actually there are no differences 8 observed that were below our de minimis threshold, for the ones 9 that were reconciled using the definition of reconciliation 10 inside the ASM. 11 Q. And you're going to be discussing this later in your 12 testimony, but the de minimis, a dollar, is that so 13 insignificant it's not worth doing any work on? 14 A. It matters for two major reasons. The first one is when 15 you're doing sampling, you're going to try to extrapolate those 16 results up to a much broader population, so you may be sampling 17 a small fraction of the transactions. So anything that you 18 discover is amplified many times over when it's extrapolated to 19 the population. 20 So how -- and remember the attribute sampling, the yes/no, 21 deciding what is yes and what is no, that makes a difference. 22 Even if it's only a dollar, it makes a difference because that 23 error rate, that yes/no error rate is going to be extrapolated 24 to the population, and it also affects how big a sample size you 25 pick and what assurance levels you can give. So it's important 1411 1 from that standpoint. 2 It's also important because not only is the dollar 3 difference used in this notion of yes/no, which is the attribute 4 sampling, it also becomes a variable on the other side, which 5 it's called variable sampling, where you sum up those dollar 6 amounts. So there's really -- going all the way back to the 7 2003 Plan, it was always clear to me from the 2003 Plan and from 8 the 2007 Plan that the fundamental design was based on attribute 9 sampling, it's a yes/no question. 10 And back in the 2003 trial there was some discussion that 11 they may eventually do a variable type sample. They may gather 12 that information. In the course of this trial it's become clear 13 that they have done that activity. In Ms. Hinkins' rebuttal 14 report, she's made it very clear, and in the administrative 15 record you can see they have gathered that additional 16 information on how much the dollar difference is. 17 Well, really, the attribute sample piece, the yes and no, 18 there's no need to set some arbitrary level for yes or no when 19 you're going to collect that information on the dollar side. So 20 it's important here, if there is any difference, it should make 21 a difference in the attribute side, whatever the size of that 22 is, because that has ramifications on the rest of the sampling 23 exercise. 24 Q. And does it make a particular difference, since the 25 statements that are being made are based on aggregate amounts as 1412 1 opposed to statements particularly to an individual Indian trust 2 beneficiary's account? 3 A. Well, here the statements that are made in the aggregate 4 are really driven off of both the yes/no question and off of the 5 dollar size question, so it makes a difference in both 6 instances. 7 Q. And therefore whether it's a dollar or 58 cents, it may be 8 significant when you're extrapolating to the entire population, 9 correct? 10 A. They need to be tracked down. As Dr. Scheuren correctly 11 pointed out, a mistake of 58 cents, you need -- if it's not 12 reconciled, you need to spend time tracking it down, 13 particularly when you're sampling, because if that transaction 14 should have been $58 or $58,000, that's going to make a big 15 difference as to how you extrapolate the implications up to the 16 population that you're able to extrapolate to. 17 Q. We've been discussing on this page the sample debit 18 transactions. Does the same concern exist with regard to 19 sampled credit transactions? 20 A. It does, but there's an additional layer to the flaw here 21 in thinking. And that is, we read here there are 11 22 underpayment differences totaling 341, and 25 overpayment 23 differences totaling 853. And then the conclusion is the net 24 overpayment was 512. 25 And this is another difference that Dr. Hinkins and I have. 1413 1 And this goes directly to my notion of sample design. It's 2 irrelevant, the last part of that sentence is irrelevant to an 3 individual account holder. It makes perfect sense in estimating 4 exposure from -- liability in a litigation setting, but if I'm 5 the individual that was underpaid $341, just take this example, 6 if someone else was overpaid $853, that's irrelevant to me. You 7 can't offset those transactions. 8 And the sample design and what was reported in the sampling 9 really gets this notion of the net. That's the way it's 10 reported by the accountants, that's the way it's reported in the 11 2007 Plan, and that netting is inappropriate in the context of 12 trying to answer the question for individual account holders. 13 Q. Is it your understanding, then, that at the completion of 14 the statistical sampling exercise, that the statement that would 15 be made to the account holder who is underpaid $341 is that his 16 stated balance is more likely than not accurate, and he would 17 not be told he was underpaid $341? 18 A. It would have to bear itself out, how the HSAs were 19 developed. But from what I've seen so far, it would be that 20 your recorded transactions are probably right. They might be 21 off by 1 percent, 4 percent, depending on credits or debits. 22 But the individual that was underpaid the 341, from what I've 23 seen so far, the only answer would be it's probably right within 24 about 1 percent. 25 Q. Can this concern that you've discussed be cured with a 1414 1 larger sample size? 2 A. No. All it would do is it would make the estimate of the 3 underpayment or the overpayment, and if they go the next step of 4 netting it, it would make each one of those a little more 5 precise, but it's not going to solve the underlying problem of 6 the inability to say something about the individual account 7 balance. 8 Q. And would that information enable the defendants to confirm 9 that individual transactions were accurate? 10 A. I'm not sure I follow that question. 11 Q. Okay. If you have a mean overpayment and mean underpayment 12 associated with a posted transaction on two electronic systems 13 and you expanded the sample size, you could not confirm based on 14 that information that the individual transaction was accurate, 15 could you? 16 A. You wouldn't have any basis. 17 Q. It provides nothing for you, does it? 18 A. Correct. 19 Q. I'd like to move on to problem No. 4, which you've 20 described as inappropriate sampling unit. Can you please 21 explain what you mean by an inappropriate sampling unit? 22 A. The issue that comes to bear here is we're trying to 23 understand what happened in accounts. Inside of the accounts 24 there are transactions. 25 I think that DOI and certainly NORC recognized early on 1415 1 that sampling accounts would be problematic, because there's a 2 likelihood, a reasonable likelihood that there would be an error 3 in one of the transactions in the account. So as a matter of 4 fact, early on in this process, when there were discussions 5 about sampling accounts, sampling transactions, and they were 6 weighing out the strengths and weaknesses, it was specifically 7 articulated that even if you have very, very low error rates in 8 the transactions, which is really contrary to what you would 9 assume given all of the other reports from accountants and GAO, 10 but if you assume it was going to be a very low transaction 11 error rate, by the time you lumped all of those transactions 12 into an account, what you're going to end up with was a high 13 likelihood that you're going on have the account balance 14 misstated. 15 So looking at the transactions, let's say you assume 16 there's only 1 percent error rate. Given the average size of 17 these accounts, by the time you look at each account and say 18 okay, there's a 1 percent chance that transaction's wrong, 1 19 percent chance that transaction's wrong, by the time you get to 20 the end of -- all the transactions in that account, even 21 assuming a 1 percent error rate, you would say you can be 75 22 percent sure that account balance is misstated. 23 If the error rate goes up at all, to 2 percent or 3 24 percent, effectively you're a hundred percent sure that every 25 account balance is misstated. So it's the wrong unit to be 1416 1 looking at. You shouldn't be looking at transactions. Really 2 what should be done here, if you were going to go through this 3 sampling exercise, is it should be done on an account basis. 4 But the lack of information really tied their hands and they 5 weren't able to do that. 6 Q. Let's take a look at an exhibit from the administrative 7 record, and it's identified as Bates No. 11501. Do you recall 8 reviewing this document? 9 A. Yes, I do. 10 Q. As you can see, this is a November 24, 2002, document 11 that's identified as draft from the Office of Historical Trust 12 Accounting, and the headnote is "Statistical sampling strategy," 13 and underneath that is "the scientific logic of using sampling 14 approaches." Is that correct? 15 A. Yes. 16 Q. I'd like you to turn to what is Bates page No. 4 of this 17 document, and I'd like you to focus in on the second paragraph. 18 It's highlighted. What you've been discussing with regard to 19 the concern about error rate in an account versus error rate in 20 the transaction, is the highlighted section one of the concerns 21 that you've had? 22 A. Yes. This articulates exactly what I was talking about. 23 It assumes a much, much -- a tenth of the error rate I was just 24 talking about. Here assuming that 0.1 percent of the 25 transactions are wrong, the account error rate would explode 1417 1 upwards of 30 percent. The example I gave was if you assume it 2 was 1 percent error rate, it would balloon up to 75. So this is 3 exactly the issue that I was talking about, and it was 4 identified fairly early in the sampling exercise, as we see 5 here, by the Office of Trust Accounting document. 6 Q. So is it fair to say that if the goal was to substantiate a 7 near zero error rate, sampling accounts would not allow you to 8 reach that goal? 9 A. You could establish a very small error rate for the 10 transactions, potentially, but it would not result in a high 11 likelihood that the account balances are correct. 12 Q. Now, is the determination of an error rate part of 13 Defendants' accounting efforts? 14 A. It is. And we keep coming back to this notion because it 15 underpins everything. What is the error rate? And that is 16 defined by what is the error. So it's very important to 17 understand what that error rate is and what comprises that error 18 rate as part of conducting the sampling. 19 Q. In that regard, I'd like you to turn your attention to an 20 exhibit, another exhibit from the administrative record. It's 21 Bates numbers 57-30-1. Have you reviewed this exhibit before? 22 A. Yes, I have. 23 Q. This is entitled "Managing historical accounting records." 24 It is undated. I'd like you to turn your attention to Bates 25 No. -- page No. 3. 1418 1 THE COURT: It's also unidentified, isn't it? 2 MR. GINGOLD: Yes, it is. But it was in the 3 administrative record, Your Honor. It's identified with an HTA 4 Bates stamp number. We assume it's something that was either 5 considered or relied on in making a decision because it was part 6 of the record. 7 THE COURT: All right. Go ahead. 8 BY MR. GINGOLD: 9 Q. In the middle of the page on Bates No. 3, we have 10 highlighted a section. And you've discussed the problem of 11 missing records, correct? 12 A. Yes. 13 Q. And you've discussed the errors associated with that. 14 A. I have. 15 Q. The highlighted section of this document specifically 16 states, "This warns us that we need to have a strategy that 17 resolves the missing record problem for disbursements in a way 18 that does not treat the failure to find the supporting 19 documentation as an error in the account." 20 Now, what is your understanding of that statement? 21 A. Well, this goes to the notion of the sample size 22 determination and what really the exercise is trying to 23 accomplish. At this point in time, it's referencing this 1972 24 to '92 tribal reconciliation. This was the reconciliation I 25 understand that was performed by Arthur Andersen. 1419 1 I've reviewed that tribal reconciliation. And the tribal 2 reconciliation found that even after making a whole series of 3 amendments to the standard by which they were, quote, 4 reconciling transactions, I believe they started out at GAAP and 5 GAAS standards, and after a series of nearly a hundred 6 modifications to their work order, at the end of the day they 7 arrived with what the BIA instructed them was the standard to 8 which it should be reconciled. 9 Even after lowering that standard all the way down, they 10 still had a substantial proportion of the documents that they 11 couldn't reconcile, that had missing supporting documentation. 12 If that was the case, and it was something in the 12 or 13 13 percent neighborhood that they could not reconcile, that would 14 be a huge implication on the sampling exercise, because if you 15 assume a 13 percent error rate, if you treat all of those 16 unreconciled as errors, then the sample size grows -- well, not 17 even proportionally, it would grow even faster than that. 18 There's a large increase in the required sample size, and 19 there's potentially a decrease in the assurance rates that would 20 be able to be provided. 21 So I think it's correctly identified here that if they're 22 going to treat these missing documents and unsupported documents 23 as errors, they're going to have -- it would have to be a 24 substantially different sampling exercise than what was 25 conducted. 1420 1 Q. Is it fair to say they wouldn't be able to achieve a near 2 zero error rate? 3 A. Correct. If you don't find a way to ignore those as 4 errors, you're not going to get close to a zero error rate; your 5 expectation going in would be something in the 12, 13 percent 6 area. 7 Q. Is it your understanding that this particular concern on 8 how to treat missing documents vis-a-vis error rate is something 9 that was actually underscored by the Defendants in the 10 administrative record? 11 A. Yes, it was. 12 THE COURT: If there's no objection to that question, 13 I guess I'll let it stand, but this is the same document you 14 showed to Dr. Scheuren and he couldn't identify it. So I'm not 15 taking very much from this. Go ahead. 16 BY MR. GINGOLD: 17 Q. Let's go to Bates page 10 of the same document. And if we 18 can turn to the second paragraph, which is the first highlighted 19 paragraph on this page, you will note that that same warning 20 with regard to treating missing documents is restated, isn't it? 21 A. Yes, it is. 22 Q. And you have reviewed this entire document, have you not, 23 Mr. Duncan? 24 A. I have. 25 Q. Now, one of the things that you've pointed out is documents 1421 1 are resolved depending on coded numbers, correct, 1, 2, 3, and 2 4 -- or reconciliations are resolved -- 3 A. Correct. There were four codes for the reconciliation 4 efforts. 5 Q. All right. And what is your recollection of code No. 1? 6 A. It changes modestly over time, but the consistent 7 definition of an accounting code 1 is it has a directly 8 supporting document, it has been recomputed using known 9 Department of the Interior methods, or it has been substantiated 10 with third-party documentation. 11 Q. And what about a code No. 2? 12 A. Code No. 2 is this term, alternative procedure. I've 13 really, despite some extended efforts to try and find something 14 as to what that means, I've not seen anything other than it's 15 called an alternative procedure. 16 Q. As a matter of fact, you've reviewed portions of the 17 administrative record, isn't it true? 18 A. The large majority of the administrative record I've 19 reviewed personally. 20 Q. Did you find anything in your review of the administrative 21 record which explained the alternative procedures in detail? 22 A. Not in any detail that I could decipher what exactly was 23 the guidance to the individual accountant that's doing the 24 reconciliation. 25 Q. If we can go to the middle of this page, you'll see a 1422 1 highlighted A, reconciled with directly relevant documents 2 without discrepancy, and you'll see A, B, C, D, E, 3 unsubstantiated. Is A consistent with your understanding of a 4 1-rated reconciliation? 5 A. Well, accounting code 1 would be both A and B. There's 6 sort of two steps to it. The first is the reconciliation. Can 7 we tie it back to some document that substantiates it's either 8 reconciled yes or no, and then the second question is is there a 9 discrepancy in that reconciliation. Did we find a supporting 10 document that's different or did we find supporting documents 11 that support it. So accounting code 1 deals with A and B. 12 Q. So accounting code 1 even includes documentation which may 13 be in conflict with the posted entry, correct? 14 A. From the standpoint of it being reconciled, yes or no, it 15 is. 16 Q. And if we look at C and D on this page, "inferred 17 reconciliation, strongly persuasive," and "inferred 18 reconciliation, persuasive to some extent," do you have an 19 understanding of what C and D mean? 20 A. Those map in some degree into accounting code 2 and 21 possibly 1. 22 Q. But again, you don't have -- you haven't been able to 23 review any detail which explains the accounting codes, have you? 24 A. Other than the ASM, but that does not tell me definitively, 25 and the information about the results of the sampling don't tell 1423 1 you even whether or not, for accounting code 1, were they 2 reconciled with a directly relevant document, or were they an 3 inferred reconciliation. You can't tell in that bucket that's 4 grouped as reconciled code 1 which of those happened. 5 Q. And in code 2 sound judgments are permitted, correct? 6 A. I believe code 1 and code 2, judgment is permitted. 7 Q. On the paragraph underneath the A, B, C, D, E descriptions, 8 there seems to be an explanation of A through E. Did you review 9 that before? 10 A. I did. 11 Q. What is your understanding of that explanation? 12 MR. WARSHAWSKY: Objection. This document speaks for 13 itself. 14 THE COURT: Thank you. I'm going to sustain the 15 objection. I tried to -- I mean, this is just a document that's 16 found in the administrative record. Nobody quite knows what it 17 is. I'm more interested in this witness's opinion than what he 18 thinks of some other paper that's in the administrative record. 19 BY MR. GINGOLD: 20 Q. Do you believe reconciliation should be performed without 21 direct source documents? 22 MR. WARSHAWSKY: Objection. No foundation for the 23 witness -- 24 THE COURT: I'll allow that question. I'll allow 25 that. 1424 1 THE WITNESS: Ultimately that's what underpins all of 2 the sampling exercise, is what is reconciled and what is not. 3 And as a statistician, if it were tasked to me to try and 4 perform this particular sampling exercise, it makes a huge 5 difference whether or not something is reconciled with a 6 directly supporting document, or whether or not it has been used 7 through some judgment or recalculated or something like that. 8 So I would not be comfortable preparing opinions or doing 9 an extrapolation where you don't provide to the consumer of this 10 information what the difference was, which ones were reconciled 11 where there was actually back to a source document that says, 12 and it matched. Which ones were recomputed? Because quite 13 frankly, if I was looking at one that was recomputed, I'm not 14 really sure that you would expect to find any error. Or one 15 that is an alternative procedure. I don't know what an 16 alternative procedure is, but I know what it's not. It's not 17 linked back to a source document, directly tied to that source 18 document. It's not a third party document. It's not inferred 19 from the recomputation of the efforts. 20 So everything in accounting code 2, what that means is it's 21 something other than that, but I wouldn't really have any idea 22 why you would expect to find an error under an alternative 23 procedure. 24 So when all of those get lumped together for the 25 calculation of the error rate, in my mind that makes anything 1425 1 based on that error rate meaningless, because you just don't 2 know how you got to this notion of there being a 1 percent error 3 rate. 4 BY MR. GINGOLD: 5 Q. And specifically, is what you're saying that you as a 6 reviewer of the information would not be able to make a decision 7 as to what is an error in a particular transaction? 8 A. Correct. The entire sampling is based on this notion of 9 the error rate, and if you don't have any understanding of what 10 constituted the error, or have any confidence that an error has 11 a meaning other than it could be any of these other procedures 12 and not be an error, it would be nearly impossible to make any 13 informed decision on that basis. 14 Q. I'd like to bring to your attention another document from 15 the administrative record. It is identified as Bates No. 16 60-12-1. This is a September 22, 2003, document entitled 17 "Accountant's report on the reconciliation of the Eastern Region 18 land-based non-interest individual Indian money transactions." 19 Have you reviewed this document before, Mr. Duncan? 20 A. Yes, I have. 21 Q. I'd like you to turn your attention to Bates page 4, and 22 I'd like you to look at this through page 10. And as you can 23 see on Bates 4, you have categories which are identified as 24 trans ID, transaction amount, throughput transaction amount, 25 reconciled transaction amount, total variance, and accounting 1426 1 code. Do you have an understanding of what this is? 2 A. I do, and this document was very important in creating and 3 drafting our opinions. This is the only information that was 4 provided as part of the administrative record where we actually 5 see what the accountants did in the process of doing the 6 reconciliation. 7 So what we have here is for the Eastern Region, this 8 document goes down for the 289 transactions that were sampled in 9 the Eastern Region, and what we see going across the line is 10 there's an identifier, the 001, that just identifies the 11 transaction, we can see that in the electronic ledger, this was 12 a $636.82 transaction. This was reconciled, if you go all the 13 way to the right column, you can see this is an accounting code 14 1. So that tells us that this was either directly reconciled, 15 recomputed, or subject to third party documentation. It was one 16 of those three things happened. And that reconciled to an 17 amount of $636.82 with no variance. 18 So for this particular transaction, it's possible, although 19 we can't tell, it's possible that this had an actual transaction 20 that would show us exactly this is the right amount. It was a 21 lease collection or check disbursement, depending on positive or 22 negative, but there's this notion that you can see this is 23 exactly how much came in. That's a possibility. It's also a 24 possibility that it was just recomputed using some known 25 Treasury method or Department of Interior method. 1427 1 Now, importantly, though -- maybe if I can just have you 2 pan down. 3 Q. Drop to let's say trans ID 48. 4 A. Good. That's one example. So we can actually see here 5 transaction 48, it's in the electronic ledger at $162.23. It's 6 also in accounting code 1. So what happened here is there must 7 have -- however it worked in accounting code 1 -- we don't have 8 the work papers. I can't see whether or not there is a check or 9 there is a lease income. All we know, we have this piece of 10 information. But it tells us that somehow they established that 11 this particular account was off -- it's only off by a penny, but 12 this transaction was off by one cent. 13 Now, that, Your Honor, perhaps to put some context to what 14 we were talking about before, but in answering a question in an 15 attribute sample, my position would be that's an error, that's a 16 mistake. It's only a penny, but that penny would be caught in 17 the second part of this, which is the variable calculation. We 18 can gather that up to quantify the magnitude, but it was 19 recorded incorrectly. 20 For purposes of the Eastern Region, though, that 21 transaction was concluded that there was no error, because it 22 fell below the de minimis rule. So this, I think, certainly 23 helped us understand when we were looking at this, exactly what 24 was happening at the individual reconciliation level. 25 So in the Eastern Region, you'll recall, Your Honor, it was 1428 1 NORC's conclusion that there were no errors. There's a series 2 of I believe 12 or 13 -- maybe 13 of these same kinds of 3 mistakes that are off by very small amounts, but at the end of 4 the day, NORC concluded there are no errors in this 5 reconciliation at all. 6 Q. And let's turn to Bates No. 5. And if we could focus in on 7 the top half of this so Mr. Duncan can read this. 8 You see there's additional variances of a penny and two 9 cents on this particular page, correct? 10 A. Yes. 11 Q. And again, you're unable to determine how that was 12 determined because you don't have the work papers, correct? 13 A. That's correct. We do know that these are accounting code 14 1. That tells us something about how it was reconciled, but 15 nothing specific. 16 Q. Let's drop to the bottom half of the page. Were there 17 additional variances identified, each of which is only a penny, 18 correct? 19 A. Correct. 20 Q. I'd like to turn to page 6. If you see on page 6 there are 21 additional transactions which are identified as a penny, 22 correct? 23 A. Correct. I believe on this page there's also some 24 negatives. There's positives and negatives. 25 Q. Does it make a difference if it's a positive or negative 1429 1 variance? 2 A. Well, this goes to the notion of the netting. At the end 3 of this report, you'll see that the net variance is provided, 4 and it does. It nets out the positives and negatives between 5 these transactions. 6 Q. What I'd like you to turn your attention to now is Bates 7 No. 9. And let's just focus on the top half of this first so 8 it's more easily readable. You note that these reconciliations 9 are done in accordance with accounting code 2? 10 A. Correct. There are I believe 42 that were accounting code 11 2 in this sample. 12 Q. But there's no variance? 13 A. I wouldn't expect there -- I don't know what the 14 alternative procedures are, but again, I know what they're not, 15 so I wouldn't expect there to be a variance here under 16 accounting code 2. 17 Q. And why wouldn't you expect a variance in accounting code 18 2? 19 A. Well, it's not linked to a directly supporting document, 20 it's not recomputed, whatever the recomputation means, and it's 21 not a third-party source; it's some alternative procedure. So 22 my guess is it's going to be highly unlikely under this 23 alternative procedure, whatever those methods ultimately turn 24 out to be, it's going to be highly unlikely that you're going to 25 be able to contradict the dollar amount with the alternative 1430 1 procedures; ergo I wouldn't expect to see an error. 2 Q. I'd like you to turn to Bates 10. And if you look at the 3 reconciliation columns above, what does that tell you? 4 A. Well, these are all accounting code 4, and I'm sure, 5 Your Honor, you've seen -- the definitions are actually provided 6 on the bottom. Transaction 4 is there's no evidence that can be 7 found, no alternative analyses, no alternative procedures. 8 These are simply ones that there was no way to reconcile. 9 Now, there's a little bit of a twist in the Eastern Region 10 because it was done early as part of the LSA, and the 11 transactions, the $105 to the $200, all of those transactions 12 are more than a hundred dollars. Here the certainty stratum 13 that was utilized was not a hundred thousand dollars and 14 greater, but it was a hundred dollars and greater. So there was 15 an attempt to reconcile all of these as well. 16 Ultimately it turns out these six transactions were not 17 able to be reconciled under anything, so they were lumped into 18 accounting code 4. And as you can see, there's no variance 19 associated with it, again, because you can't reconcile it, there 20 would be no expectation of being able to calculate a variance. 21 Q. And is it your understanding there's a zero error rate as 22 determined by NORC in the Eastern Region? 23 A. That's correct. 24 Q. Now, are you aware, other than -- you've discussed the fact 25 you haven't seen work papers which help explain how the 1431 1 accounting codes have been employed, correct? 2 A. They would be very helpful in understanding what 3 constitutes these different codes. 4 Q. Is there any other evidence in the administrative record 5 that you've reviewed that helps you understand what the error 6 rates are? 7 A. Well, there's the inclusion at the beginning of the Eastern 8 Region, there's a discussion as to what the error rates -- you 9 know, that there were no error rates -- there was a no error 10 rate conclusion with respect to this particular project with the 11 Eastern Region. 12 Q. And let's turn on this exhibit to Bates page No. 2. Is 13 this what you're referencing on the bottom of the page? 14 A. Yeah. This conclusion, I think it rolls to the next page, 15 but this was the basis for what was then provided in some of the 16 summary information about the national sample, as it related to 17 the Eastern Region. 18 So there were a total of 289 transactions. They said they 19 directly reconciled 241 of those, or they were able to directly 20 reconcile it by using directly relevant documents. Again, what 21 that means is, those were all accounting code 1, and it tells 22 you what the three different definitions of accounting code 1 23 are here that I've articulated for the Court. 24 Q. And then it goes on on the bottom of the page, carrying 25 over to Bates 3, dealing with 42 transactions which were 1432 1 reconciled, correct? 2 A. Correct. 3 Q. Using alternative procedures, correct? 4 A. Correct. It tells me nothing more than it was in fact an 5 alternative procedure. And this again links to what we saw at 6 the bottom of that last table, which is the definition of the 7 accounting codes. 8 Q. And finally, at the end of this explanation by Deloitte & 9 Touche it talks about six transactions which Deloitte was unable 10 to reconcile, correct? 11 A. That's correct. 12 Q. What does this tell you? 13 A. Well, again, the error rate, if I were designing this as an 14 attribute sample and I was going to collect the information from 15 the variable sample anyway, then I would think that any error 16 should be counted as an error, because it's going to be 17 extrapolated up to a large population in the electronic ledger 18 era. And that information indicates, based on the number of 19 errors we saw there, it's going to be something in the 6-1/2 20 percent range, would be the errors in that particular instance. 21 But by defining an error as having to reach a certain 22 level, you're able to reach -- DOI is able to reach a lower 23 conclusion about the error rate, therefore draw a much smaller 24 sample size. 25 Q. And what is your understanding of what NORC actually stated 1433 1 with regard to the Eastern Region? 2 A. My understanding is that they concluded that they 3 reconciled, of the 289, they're 98 plus percent certain that 4 there's a very low error rate based on the fact that they found 5 no errors. 6 Q. Let's look at document 51-4 at 4. You've reviewed this 7 document, haven't you, Mr. Duncan? 8 A. Yes, I have. 9 Q. And this is a statistical evaluation of preliminary Eastern 10 Region sample results, March 2004, by NORC, correct? 11 A. Correct. This is a summary of the study that we just 12 looked at by Deloitte & Touche. 13 Q. Let's go to Bates 4. Let's focus in on the highlighted 14 portion of this page. What does this tell you, Mr. Duncan? 15 A. Well, this is what we talked about this morning, is this 16 notion that because of the way the error rate was defined, that 17 NORC can say that no errors were found. And on that basis they 18 can make the kinds of statements that you see ahead of that, at 19 the beginning of that sentence. 20 Q. And so no errors were found, basically, notwithstanding the 21 issues that you've just discussed for the last several minutes, 22 correct? 23 A. Correct. 24 Q. Now, this is only the Eastern Region, though. Why is this 25 important? 1434 1 A. Again, there's nothing else in the administrative record 2 that allows us to do this level of detail, to actually look 3 behind the curtain. This is the only part of that study that 4 allowed us to do so. 5 Q. And does it also discuss the approach that NORC has been 6 taking with regard to how it is reconciling posted transactions, 7 or not? 8 A. Well, NORC's not doing any of the reconciliation work. 9 That's all done by the accounting firms. NORC designs the 10 sample, provides the sample list, and then the accounting firms 11 go through the process of doing the reconciliation work. 12 Q. What about the plan itself, the historical accounting plan? 13 Let's go to Bates 33-2-14. Now, this itself, if you look at the 14 middle of the page, this deals with tribal reconciliation, 15 correct? 16 A. Correct. 17 Q. And you're talking about error rates generally with regard 18 to NORC's work, correct? 19 A. Correct. 20 Q. And tribal reconciliation work was used as a basis for 21 taking the approach that was done in the 2007 Plan, correct? 22 A. At least in part, yes. 23 Q. Now, you also discussed the fact earlier that the tribal 24 reconciliation report had an error rate that was significantly 25 greater than 1 percent, didn't you? 1435 1 A. Well, the tribal reconciliation work had an unreconcilable 2 rate in the area of about 13 percent, and this is being 3 pulled -- I believe this comes out of the 2007 Plan directly. 4 And this goes to the notion of saying, well, doing the 5 historical accounting, we need to make an assumption. Before we 6 start, we need to make an assumption about what's the error rate 7 going to be. 8 And when they looked at things, like they said here we 9 looked at previous work like the tribal reconciliation project, 10 if you looked at that tribal reconciliation project and your 11 thinking was if a transaction is unreconciled, I will treat it 12 as an error, then the assumption you would make about the 13 underlying error rate is it would be something in the 13 percent 14 area. 15 But in fact, that's not the assumption that they made. The 16 assumption that is made, and it's all throughout the 17 administrative record, the assumption that was made was it would 18 be a very low error rate. And in order to make that assumption 19 based on what they cited in the plan, you must have made the 20 assumption that an unreconciled transaction is not an error. In 21 other words, if there's no supporting document, it's okay. 22 That's the only way that you could cite that study as a premise 23 for a very low error rate. You have to assume that unreconciled 24 is not an error. 25 Q. Do you believe it's reasonable to assume a 1 percent error 1436 1 rate based on the information in the tribal trust reconciliation 2 report? 3 A. Base on the tribal reconciliation report, based on the 4 paragraph 19 work, based on the Rosenbaum work, based on 5 everything that I have observed, I don't think it's reasonable 6 to start -- if you're going to treat unreconciled as an error, 7 it's not a reasonable basis to conclude that it's going to be a 8 very low error rate. There's just too much missing information. 9 Q. So how meaningful is the statement with regard to the 10 probability that transactions are accurate and balances are 11 correct if in fact the assumption is unreasonable? 12 A. Well, if it's premised on the notion that in order to 13 achieve an error it has to be not all the things that we talked 14 about, it can't be accounting code 1, it can't be accounting 15 code 2, in order for it to be an error it has to reach this 16 notion of there being some contradictory underlying document, it 17 really makes any of the implications to try and say something 18 about the accuracy and completeness of the account transaction 19 listing and the account balances, it really makes anything drawn 20 from that meaningless. 21 Q. You were also asked to rebut, Mr. Duncan, the expert 22 reports of Drs. Lasater, Scheuren and Hinkins; is that correct? 23 A. That's correct. 24 Q. Let's start with Dr. Lasater. Based on your recollection, 25 what were Dr. Lasater's opinions that he expressed to this 1437 1 Court? 2 A. Dr. Lasater's opinions really focused on the expectation 3 for how the paper ledger era would be executed. 4 Q. Just the paper ledger era? 5 A. As it was expressed to the Court. In his written opinion 6 he had some statements that were very similar to what 7 Dr. Scheuren had been saying. So as I read it, the Judge only 8 wanted to hear on the part that was not duplicative. 9 Q. If Dr. Lasater stated that the proposed methodology for 10 testing transactions in the paper ledger era is reasonable, 11 would you agree with that? 12 A. Absolutely not. 13 Q. Why not? 14 A. Well, first of all, I don't know what basis Dr. Lasater has 15 for reaching that conclusion. I've reviewed the administrative 16 record, I've reviewed the NORC reports. There is precious 17 little information about what exactly is going to be done in the 18 paper ledger era. So I don't think there's an adequate basis to 19 reach the conclusion that it's reasonable. We don't know what 20 it is yet. 21 Q. Did Dr. Lasater provide the basis for his opinion in that 22 regard? 23 A. Dr. Lasater indicated that he had had some discussions with 24 NORC about how they were going to do that, but nothing really 25 materially beyond what we knew from the administrative record. 1438 1 Q. Did he reference any documents that supported that opinion? 2 A. No. 3 Q. If Dr. Lasater stated in his opinion to this Court that the 4 proposed difference testing to evaluate statistically whether 5 the paper ledger and electronic era transactions were processed 6 by the same quality of bookkeeping is sound, would you agree 7 with that opinion? 8 A. I definitely would not agree with that, on several levels. 9 The notion -- the hypothesis that Dr. Lasater's putting forth 10 and that is vaguely described in the plan is that whatever was 11 done over here in the electronic ledger era, what we're going to 12 do is just go look in the paper ledger era, and if in fact 13 there's no material difference between what we see in the paper 14 ledger era versus the work that was done in the electronic 15 ledger era, then we'll assume that those two populations are the 16 same. 17 Well, every bit of information I've reviewed in the 18 administrative record indicates that that is not the case. 19 There's not even a reasonable basis to start with that 20 hypothesis. 21 The only way that that hypothesis would potentially make 22 sense is if the criteria upon which you compare the electronic 23 ledger era to the paper ledger era is the same criteria, with 24 this very, very high hurdle to create an error. Something where 25 you would have almost no expectation of finding the error. You 1439 1 might be able to reach the end conclusion that they're the same, 2 but it's only because that which you are seeking to detect would 3 not be detectable by the systems that you're using. 4 Q. Does the FTI September 30, 2007, report provide the support 5 for Dr. Lasater? 6 A. No. 7 Q. And why not? 8 A. There's just not enough information in that report to 9 determine that that's definitively the way that it's going to be 10 done. 11 Q. I'd like you to turn your attention to another exhibit from 12 the administrative record. It is identified as 55-17-19. Have 13 you reviewed this report? 14 A. Yes, I have. 15 Q. I'd like you to turn to page 37 of this report. And I'd 16 like you to focus on the second bullet point, which is 17 highlighted on this page. It states: "A review of the 18 paragraph 19 search indicates that records recovery diminished 19 from the 1930 to 1950 time frame and fell off before 1930." 20 What is your understanding of that statement? 21 A. Well, this is just one piece of information. When I said I 22 reviewed the administrative record and it was clear to me that 23 the expectation should be that the records in the paper ledger 24 era are significantly worse than the electronic ledger era, this 25 is simply something -- I believe this was presented by 1440 1 Mr. Edwards. This is simply documenting that fact, that the 2 records get worse the further you go back in time, which is not 3 a particularly controversial statement. 4 Q. So when Dr. Lasater made or offered his opinion, he didn't 5 address the concern of the diminishing availability of records, 6 did he, as you go further back in time? 7 A. He did not. 8 Q. Did Dr. Scheuren? 9 A. I don't believe that he made any specific comments or 10 opinions about how they were going to deal with this particular 11 issue. 12 Q. So if the observations were made by Dr. Scheuren and 13 Dr. Lasater that the two eras, as they define paper and 14 electronic, are substantially similar, you see no basis to 15 support those opinions, do you? 16 A. I think the hypothesis itself is a waste of time to even 17 bother to test. Everything would be contraindicated here. The 18 hypothesis should be that they're fundamentally different based 19 on everything that we've seen. 20 Q. And when you say everything, do you mean audit reports? 21 A. Audit reports, reconciliation reports, GAO reports, I mean 22 reports to Congress, everything I've seen indicates that the 23 records drop off substantially as you move back in time. And 24 again, it's because there's missing documents. There's records 25 that have been destroyed. 1441 1 Q. Do you recall any discussion by Ms. Herman about how the 2 record and bookkeeping systems among the various regions and 3 agencies were different over time? 4 A. Yes. She testified that they varied over time and between 5 agencies. 6 Q. So it would be very difficult in your opinion to reach the 7 conclusion that the paper records era and the electronic records 8 era are sufficiently similar to draw inferences from the 9 electronic era to the paper era, correct? 10 A. I think that's a fair statement. 11 Q. Now, Dr. Scheuren indicated that -- didn't Dr. Scheuren 12 indicate that no additional analysis or testing is necessary in 13 that regard? 14 A. Dr. Scheuren indicated that there was no additional 15 analysis done in the electronic era. 16 Q. Did he indicate whether or not additional analysis is 17 necessary? 18 A. In the electronic era? 19 Q. Correct. 20 A. He indicated that it was not. 21 Q. Now, did Dr. Scheuren take into consideration that the LSA 22 project was designed to estimate risk exposure, not transaction 23 accuracy? 24 A. No. Again, the notion of the application of the faulty 25 sample design was not addressed as a rebuttal to my opinion by 1442 1 Dr. Scheuren. 2 Q. And did either Dr. Lasater or Dr. Scheuren take into 3 consideration in their opinion the fact that the target 4 population is different from the sample population due to 5 missing and omitted data? 6 A. I don't believe that either one them specifically addressed 7 that. It was really addressed by Dr. Hinkins. 8 Q. Do you know whether or not anyone has been tasked with 9 establishing the accuracy of account balances? 10 A. I don't know if anyone -- I don't know of anyone that has 11 been. I do know that NORC specifically stated they were not 12 tasked with designing a sample that would address that issue. 13 Q. And that was in Dr. Hinkins' rebuttal report, wasn't it? 14 A. That's correct. 15 Q. What effect, if any, does that have on your view of the 16 adequacy of the statistical sampling plan? 17 A. Well, I think fundamentally Dr. Hinkins and I agree on 18 several topics. This is one of them. My criticism, my concern 19 during my time in 2003, and at my preparation for the trial here 20 in 2007, has been that you can't take this kind of transaction 21 sampling and apply it to accounts. And the rebuttal point on 22 that issue from Dr. Hinkins was that NORC had never been asked 23 to do a sample design that would address accounts. 24 So I think fundamentally we're in agreement on that point. 25 What has been done in the LSA sampling will not address 1443 1 accounts, account balances. 2 Q. But if the LSA sampling was designed for purposes of 3 limiting liability or error, it could be used as a sound basis, 4 could it not? 5 A. If it's limited to the electronic ledger era, and limited 6 to the documents that were available for sampling, then yes. 7 Q. And similarly, couldn't NORC's LSA results for purposes of 8 the litigation support accounting be considered as having an 9 appropriate measure of error? 10 A. I'm sorry, I couldn't hear you over the coughing. 11 Q. With regard to NORC's LSA results, the litigation support 12 accounting results, and its use of error in that regard, would 13 it be inappropriate use of error in the LSA that it is using 14 versus what we're talking about, the establishment of accurate 15 account balances? 16 A. No. Again, I think the error rate is too narrowly defined 17 to be useful even in that context. 18 Q. And does a qualitative meta-analysis report assist 19 Dr. Scheuren in his opinions with regard to the appropriateness 20 of drawing inferences from the electronic record era to the 21 paper record era? 22 A. I don't believe it provides any reasonable basis for making 23 that conclusion. 24 Q. Why is that? 25 A. Well, the meta-analysis, first of all, we obtained some 1444 1 information from the meta-analysis. There were many of the 2 things that had been referenced in the meta-analysis that were 3 not provided to us. Of the documents that were provided to us, 4 many of them had missing pages, which from our review appeared 5 to be pages that were material, that would have potentially 6 influenced the way that that meta-analysis was reviewed. 7 But in any event, the documents that we were provided and 8 did look at, many of them were internally inconsistent with the 9 conclusions that had been reached in the meta-analysis. So I 10 don't feel that the meta-analysis forms a reasonable foundation 11 or basis upon which to make a conclusion that the paper ledger 12 era is the same from a bookkeeping standpoint as the electronic 13 ledger era. 14 Q. Now, I'd like you to look at the fourth demonstrative you 15 have on the original exhibit, where you actually identified as 16 you understood it the NORC solution in redefining the target 17 population. Do you recall preparing this demonstrative? 18 A. I do. 19 Q. Could you please explain it? 20 A. Well, this is another area where I think Dr. Hinkins and I 21 fundamentally are in agreement. I identified this issue prior 22 to Dr. Hinkins issuing her report, that the target population, 23 that there were things missing and omitted from the target 24 population. And Dr. Hinkins in her rebuttal clarified that in 25 fact that was not what they were calling the target population. 1445 1 They, NORC, had said the target population is actually the 2 population from which we drew the sample. 3 And if that's the way you define the target population, 4 then my criticism falls away. You can actually draw an 5 inference from the population from which you've sampled. So to 6 the extent that NORC is now clearly delineating that the target 7 population is actually only that population from which the 8 sample was drawn, then I think everything's fine. 9 Q. It is fine, but it doesn't allow statements with regard to 10 the accuracy of account balances, does it? 11 A. Well, it's fine. It's just irrelevant. I mean it's 12 irrelevant to the real target population, which is all of the 13 account holders. 14 Q. And that's all the past and present individual Indian trust 15 beneficiaries, is that your understanding? 16 A. That's my understanding. 17 Q. Does the target population sample allow NORC or the 18 defendants to make a statement about the account balances for 19 its new target population? 20 A. Not for the account balances. It's still going to be 21 limited to account transactions. 22 Q. And so notwithstanding that, you don't have any 23 disagreement with what Dr. Hinkins stated, correct? 24 A. Not in this regard. If you correctly limit the 25 extrapolation to the population from which you've sampled, 1446 1 that's sound statistical reasoning. 2 Q. Now, Dr. Hinkins also stated in her rebuttal that the 3 qualitative meta-analysis report is not central to the sample 4 design; is that correct? 5 A. That's my recollection. 6 Q. What is your opinion in that regard? 7 A. I think it's appropriately relegated to not being central. 8 It doesn't provide a reasonable basis, so that's the right spot 9 for it. 10 Q. Is there any other basis for Dr. Scheuren coming to the 11 conclusion that you've seen in the record that the paper ledger 12 era is similar to the electronic ledger era, other than a 13 meta-analysis? 14 A. Well, a meta-analysis I don't think is a reasonable 15 foundation to make that conclusion, and everything that I have 16 reviewed is to the contrary. 17 Q. With respect to your overall views of what is being 18 prepared here, and that is -- the error rate is one of the most 19 important issues, isn't it? 20 A. It's absolutely fundamental to the entire discussion we're 21 having. 22 Q. Do you recall reading in the transcript testimony the 23 testimony of Dr. Scheuren with respect to the error rate in the 24 LRIS tract history reports? 25 A. If this is the DataCom issue, then I believe that's -- 1447 1 Q. Let me identify for purposes of this question a document 2 from the administrative record which has Bates No. 50-2-1. Now, 3 have you reviewed this document, Mr. Duncan? 4 A. Yes, I have. 5 Q. I'd like you to turn to Bates No. 8. I'd like you to focus 6 on the statement that is highlighted. It states, "We," which is 7 NORC, "particularly emphasize our positive results in view of a 8 report on the findings of the TIME project carried out by 9 DataCom in 2000. DataCom asserts an error rate of about 30 10 percent in postings. NORC has found no evidence of such a 11 figure." 12 Is this an example of the importance of a sound definition 13 of an error rate? 14 A. Yes. This goes on to say that "such stark differences with 15 our findings are not explained by statistical variance alone." 16 That's absolutely correct. This is definitional. If an 17 error -- if many of the things were defined as an error from the 18 standpoint of DataCom but not from the standpoint of NORC, 19 that's how you can get these kinds of stark contrasts. 20 Q. And the contrasts as a result of a narrowly defined error 21 rate provide meaningless results for a trust beneficiary, 22 correct? 23 A. It's my conclusion that the way these error rates have been 24 defined, it's so difficult to clear the hurdle of being an error 25 that the error rate premise underlying all the statistical 1448 1 analysis is unfounded. 2 Q. I would like to ask you if you agree on one final opinion 3 with something that Dr. Scheuren stated. And I'd like to pull 4 up for purposes of this question Bates 57-32-1. This is the 5 straw man pretest. Do you recall reviewing this document? 6 A. I do. 7 Q. First of all, do you have any understanding of what the 8 straw man pretest is about, based on your review of it? 9 A. My review of this document indicates to me that this is an 10 attempt to -- the notion of a straw man is to put up an argument 11 that can be fought. Generally speaking that argument is a 12 fallacy. It's meant to distract. But it's really just to put 13 something up that everyone can take shots at. 14 Q. And I'd like you to pay particular attention to page 2, 15 Bates No. 2. And the last sentence in the second paragraph. 16 Dr. Scheuren stated that he prepared this document, and it 17 states, "Clearly we are not talking about the standards used in 18 a court case." 19 Would it be your understanding that the standards that are 20 being used with regard to the statistical sampling are not 21 standards that would allow a trust beneficiary to receive an 22 accurate statement of his account balances? 23 A. That is precisely my opinion. 24 Q. Would it be your opinion that the standards used -- 25 THE COURT: Let him give his opinion instead of you 1449 1 stating it and asking him to say yes or no. I mean this is sort 2 of the ultimate question. 3 BY MR. GINGOLD: 4 Q. Explain your opinion. 5 THE COURT: Thank you. 6 THE WITNESS: The fundamental premise is what is the 7 reconciliation, has something been reconciled or not, and is it 8 an error or is it not. And at the end of the day the statements 9 that are going to be able to be made from the statistical 10 sampling that was done and from the reconciliation efforts that 11 were done will not provide results that really would have any 12 meaning to the individual account holder. 13 They don't have any understanding of whether or not their 14 balance is correctly stated. There's going to be some level of 15 assurance that the transactions are probably correct, but at the 16 end of the day, as far as providing information to the 17 individual account holder on their account, I would contend, it 18 is my opinion, that the information that's being prepared as 19 part of the 2007 Plan would be meaningless. 20 MR. GINGOLD: Your Honor, our cross-examination is 21 completed. I would like to move into evidence -- 22 THE COURT: I don't think the cross-examination has 23 begun yet. 24 (Laughter) 25 MR. GINGOLD: I'm sorry. We spent a week and a half, 1450 1 Your Honor, of cross-examination. I'm sorry. The direct 2 examination is completed. We'd like to move into evidence 3 Mr. Duncan's expert report and his expert rebuttal report. 4 THE COURT: All right. They'll be received. 5 (Plaintiff Exhibit Nos. PPX 6 4284, 4484 received into evidence.) 7 MR. GINGOLD: Your Honor, one more we would like to 8 try and move into evidence, which is the demonstrative 9 identified as PPX 4486, which is compiled based on the DCV 10 information provided in the September 30, 2007, FTI DCV report 11 and testimony from Ms. Herman. 12 THE COURT: That will be received. 13 MR. GINGOLD: Thank you, Your Honor. 14 MR. WARSHAWSKY: For the record, we object. It's 15 demonstrative. It's not substantive evidence. 16 THE COURT: It'll be received. It'll be part of the 17 record. It'll be received as demonstrative evidence. How does 18 that sound? 19 MR. WARSHAWSKY: Sounds perfect. 20 (Plaintiff Exhibit No. PPX 21 4486 received into evidence.) 22 THE COURT: All right. Mr. Warshawsky, you may 23 cross-examine. 24 MR. GINGOLD: Your Honor, there's one more, I'm sorry. 25 THE COURT: Somebody get the hook. 1451 1 MR. GINGOLD: Oh, I'm sorry. Okay. 2 THE COURT: No, no. Go ahead. I'm just kidding. 3 Obviously, you smiled, though, Mr. Gingold. You have a nice 4 smile. 5 MR. GINGOLD: Thank you, Your Honor. PPX 4468, which 6 is a document entitled "The Treasury Department's role in the 7 administration of tribal trust funds." It's a Morgan Angel 8 report dated April 16, 2004. 9 MR. WARSHAWSKY: Your Honor, we object again. This is 10 a draft document. It was prepared for Treasury. It's not 11 probative or relevant to the issues. 12 MR. GINGOLD: Your Honor, all Morgan Angel -- Ed Angel 13 testified that they routinely prepare their documents in draft 14 until they're otherwise told by the Defendants to produce them. 15 THE COURT: Do we know what this is? This is a draft. 16 Everybody agree this is a draft? 17 MR. GINGOLD: No, Your Honor. We do not. They 18 routinely dub their documents draft documents. The testimony 19 is, not only before this court in this current proceeding, but 20 in the trial 1.5, Mr. Angel repeatedly responded to questions of 21 Mr. Harper that they routinely dub all their documents draft 22 documents. 23 MR. WARSHAWSKY: Your Honor, as you may recall, this 24 document was the subject of a motion that we filed. This was 25 inadvertently disclosed in discovery, and most certainly is 1452 1 considered by Treasury and Morgan Angel to be a draft document. 2 With regard to tribal -- 3 THE COURT: We had Dr. Angel up here and nobody asked 4 him what this was. 5 MR. WARSHAWSKY: It wasn't relevant to our 6 presentation of Dr. Angel. 7 MR. GINGOLD: Your Honor, it was identified in the 8 opposition to -- in the motion to suppress this document, the 9 government identified this as -- 10 THE COURT: It will be part of the record of this 11 case, but it will be assumed to be a draft unless somebody has 12 some other testimony establishing to the contrary. 13 What about 4485, your demonstrative? 14 MR. GINGOLD: We would like to move all the 15 demonstratives in, Your Honor. 16 THE COURT: All right. I'll receive 4485 as a 17 demonstrative exhibit as well. 18 MR. GINGOLD: Thank you, Your Honor. 19 THE COURT: Thank you. 20 (Plaintiff Exhibit Nos. 4468 21 and 4485 received into 22 evidence.) 23 CROSS-EXAMINATION 24 BY MR. WARSHAWSKY: 25 Q. Good afternoon, Mr. Duncan. 1453 1 A. Good afternoon, Mr. Warshawsky. 2 Q. Mr. Duncan, you've testified about a wide variety of 3 subjects this morning and this afternoon. Is that fair to say? 4 A. I think they've all been pretty focused to the issue of the 5 2007 Plan and the sampling associated with it. 6 Q. I appreciate you saying that. Principally, you're here to 7 talk about statistics, right? 8 A. Well, statistics and also to address the issue of the DCV, 9 which was provided to me in late September. It was not provided 10 before we provided our expert opinions. But my understanding at 11 the direction of counsel was that I was to analyze that and that 12 the Court would listen to my opinions and my conclusions 13 regarding the DCV. 14 Q. And that's why you provided us with Plaintiffs' Exhibit 15 4486 last night, correct? 16 A. Correct. 17 Q. All right. We'll get to that one. Let's talk about your 18 background in statistics, please. Can you put up Plaintiffs' 19 Exhibit 4284, please? This is your expert report. You remember 20 this. And specifically go to pages 43 and 44 of 79. 21 Mr. Duncan, do you recognize these two pages? 22 A. I do. 23 Q. And what is set forth on these two pages? 24 A. This is my deposition and testimony history over the last 25 10 years, I believe. 1454 1 Q. And I counted it, it looked like there were about 32 2 matters in which you've testified? 3 A. Correct. 4 Q. And of those 32, roughly 20 since you testified in the 2003 5 Cobell case; is that correct? 6 A. That's correct. 7 Q. So, in other words, in the intervening 50 months since you 8 testified in the Cobell case, you've testified as an expert in a 9 different matter about every two and a half months; is that 10 correct? 11 A. Seems about right. 12 Q. And would you identify, putting aside the Cobell matter, 13 the matters on those two pages in which you've been admitted or 14 accepted as an expert on statistics. 15 A. The final piece there, Mirage Crossings, is a statistical 16 sampling report as it relates to construction defect cases. The 17 opposing expert on that case is a Ph.D. statistician out of 18 Berkeley, I believe. 19 Q. And you've been accepted by the Superior Court of Arizona 20 in that case as a statistician. 21 A. I'm sorry, I believe you asked me which ones I'd testified 22 on. So this is a testimony in deposition only at this stage. 23 Q. All right. Okay. Can you identify any where a court has 24 accepted you to provide opinions as a statistician? 25 A. If you can zoom out and we go up a little bit. 1455 1 Q. I know you brought some papers up there. Do you have a 2 copy of your report up there by chance? 3 A. I believe I may. 4 Q. With the caveat that if there are notes in there I might 5 want to look at them. If it would assist you to review those. 6 A. This is a copy of my expert report and, given your caveat, 7 I will tell you that there are no notes. 8 Q. Okay. So feel free to refer to that if that would assist 9 you. 10 A. In the interest of trying to keep my carry-on baggage 11 light, I have omitted my resume. I'm pretty familiar with it 12 from the standpoint of needing to review it on the plane, so I 13 don't make a copy of it. 14 Q. My question was, though, whether you had been admitted as a 15 statistician in any of these matters? 16 A. Well, let me go back and look on the screen here. Go down, 17 please. I'd have to go back and look at both -- what's called 18 B&H Manufacturing, Inc., whether or not I was specifically 19 tendered as a statistician in that context, but there was quite 20 a bit of statistical testimony there. And in the Mitsubishi 21 versus Hughes Electronics case. I'd have to look whether that 22 was specifically -- I was tendered to the court in that role, 23 but that was a very central part of both of those testimonies. 24 Q. You don't remember that you were actually admitted as a 25 statistician there? 1456 1 A. I don't recall either way. 2 Q. In your selected experience summary, pages 35 to 40 of your 3 expert report, you listed 16 separate areas of experience. 4 You're familiar with those 16 areas, of course. 5 A. Yes, I am. 6 Q. Have you listed being a statistician in any of those 16 7 areas? 8 A. A lot of those areas obviously contain statistical 9 analytical work, but I have not listed straight -- statistical 10 analysis is a criteria permeates really all of the analytical 11 work that I do. 12 Q. You know, I asked you that specifically in the 2003 13 hearing, and even after testifying in the 2003 hearing, you 14 didn't update your CV to list statistics now as being one of 15 your professional experience areas, correct? 16 A. I didn't think the criticism was relevant then either. 17 Q. The answer to the question, though, is you did not update 18 your CV to say you're now a statistician. 19 A. That's correct. The statistical work that I do as embodied 20 in some of these representative cases, I don't feel it necessary 21 to clarify for the individuals that I provide my resume to, 22 beyond the fact that there's a significant component of 23 statistical work, I don't feel required to try and split that 24 out as a separate category. It permeates. 25 Q. Okay. Fair enough. Pages 40 and 41 of your expert report, 1457 1 you describe your teaching experience, and you list four areas 2 of economics. Do you remember that, sir? 3 A. Yes, I do. 4 Q. At the bottom of this page, continues on to the next page. 5 And in none of those instances do you even refer to statistics 6 as being an area of your teaching, where you teach, correct? 7 A. Again, as part of the micro and macroeconomics, there is a 8 significant component which is -- and you asked me previously in 9 my deposition if there were statistics contained in those 10 courses, and I informed you there was statistics, sampling, 11 sample design construction. Those are part of those courses 12 that I was teaching. 13 Q. But it's not a subject that you actually teach, statistics? 14 A. Separate and apart from economics, I do not. It's part of 15 the statistical analyses and survey work that I've done as an 16 economist. 17 Q. Okay. And we've got up here on the screen page 41, the 18 publications that you've authored. Any of those have to do 19 strictly speaking with statistics? 20 A. All of these would in some part have a statistical 21 component to them. A lot of the economic blue chip forecast 22 work that we do is premised upon statistical analysis work that 23 I'm doing of trends, macroeconomic and microeconomic trends. 24 The notion of intellectual property and antitrust litigation. A 25 lot of those underpinnings have statistical analytical work 1458 1 involved in them as well. 2 Q. But for example, you haven't authored a publication in a 3 statistics -- a professional statistics journal. 4 A. I haven't authored a publication in an economics journal 5 either. As an economist, working as an expert witness, that is 6 not something I regularly engage in. 7 Q. That's your profession, by the way, isn't it, being an 8 expert witness? 9 A. I would say that's probably 80 plus percent of the work 10 that I do. 11 Q. Now, one of the things that you disclosed on your CV was 12 that you're a chartered financial analyst. What's a CFA? 13 A. It's a chartered financial analyst. 14 Q. Okay. And for those of us who aren't CFAs, can you tell us 15 what that means? 16 A. It is -- a CPA is an application of an undergraduate degree 17 in accounting. Usually there's an accreditation occurred with 18 using those skills as an accountant. The CFA would be the 19 equivalent of that except it's an application of graduate level 20 skills in economics, finance, statistics, real estate, 21 securities markets and bond markets. 22 Q. Principally involved with investment management, investment 23 banking, financial analysis of stocks, bonds, correct? 24 A. No. That's a big part of it, but it's not principally 25 that. There's also a significant valuation component built into 1459 1 that. Valuing closely held businesses, things of that nature. 2 Q. It's not a title that one associates with a statistician, 3 right? 4 A. Several of my colleagues that are involved and hold the CFA 5 charter are statisticians. They have graduate degrees in 6 statistics and a lot of the analytical work they do as CFAs, 7 particularly underlying big bond portfolios and stock 8 portfolios, like pension funds, the work they do day to day is 9 as a statistician, but it's applied in the field of finance. 10 Q. You're talking about colleagues that have graduate degrees 11 in statistics, correct? 12 A. Correct. 13 Q. You don't have a graduate degree in statistics, do you? 14 A. That's correct. 15 Q. In fact, on page 42 of your expert report you describe your 16 educational background. Master's and a bachelor's degree in 17 economics, and your master's degree you list, what is it, five 18 areas of specialization? 19 A. Correct. 20 Q. And none of them are statistics, right? 21 A. No. As we've talked about in my deposition years ago, 22 econometrics is specifically the application of statistics and 23 statistical concepts in the field of economics. So I would say 24 that it is exactly that. It is statistics, just as economists 25 use it. 1460 1 Q. Well, just to cut through it, there are people who use 2 statistical analysis in their work, and that's you, right? 3 A. I certainly do. 4 Q. And there are statisticians like Dr. Scheuren and 5 Dr. Hinkins, right? 6 A. I don't know that you make a distinction. I think they 7 certainly use statistics in their work as well. 8 Q. You consider yourself to be an equivalent statistician to 9 Dr. Scheuren and Dr. Hinkins? 10 A. I don't have a Ph.D. in statistics, but from the standpoint 11 of using it on a day-to-day basis, I would say that we do very 12 similar things. 13 Q. By the way, I remember at the 2003 hearing, you joined the 14 ASA, the American Statistical Association in the middle of the 15 trial. Do you remember that, sir? 16 A. Correct, I did. 17 Q. How's that membership going? 18 A. I've let that membership lapse. It really was not very 19 beneficial. 20 Q. So you didn't go to their meeting last year in Salt Lake 21 City? 22 A. I did not. 23 Q. Now, let's talk a little bit about some of the stuff you 24 have done as an expert witness. Obviously, today you testified 25 about issues involving statistics. You've testified about a lot 1461 1 of other areas beyond just what a statistician does, or even 2 economists, right? In 2003, for example, you offered opinions 3 to Judge Lamberth about the legal application of Daubert in 4 Kumho Tire, do you remember that? 5 A. I provided testimony as it related to whether or not the 6 expert work that was done by some of the other experts in the 7 case was -- properly relied upon other experts, which I opined 8 that it did, and the use of geographic information systems, as 9 that was something that was typically relied upon by experts, 10 and in my experience I provided an opinion to the court that it 11 was in fact something that was used regularly by experts. 12 Q. In fact, it was your opinion, though, for Judge Lamberth 13 that the analysis of Mr. Fasold and the folks who worked for 14 him, that that analysis satisfied the reliability and relevance 15 standards of Daubert in Kumho Tire, correct? 16 A. Correct. 17 Q. And you're not an attorney, are you? 18 A. I'm not. 19 Q. That may not be a bad thing. But let me ask you, page 43 20 of your expert report, you talk about a case that you worked on 21 called Karron Power versus Delta Houseboat. Can you describe -- 22 that's in the U.S. District Court for the Eastern District of 23 California, right? 24 A. Correct. 25 Q. What was your opinion -- the subject matter for your 1462 1 opinion there? 2 A. This had to do with the quantification of damages to 3 Ms. Power. She was a surgeon and she had actually had her 4 finger severed in a boating accident, and it was an analysis of 5 the statistical information on projected future earnings, 6 projected for her practice, had to look at Bureau of Labor 7 Statistics, what likely her profession would be and what 8 progress, path she would make going down that road, and quantify 9 her personal injury loss as a result of that accident. 10 Q. And the opinions that you offered in that case actually 11 extended to opinions about vocational rehabilitation and life 12 care issues, right? 13 A. On a very limited basis was I involved in the life care 14 side of that. There were -- some of those inputs were required 15 for quantifying her economic loss as a physician, as a surgeon. 16 Q. In fact, your report was the subject of a motion in limine 17 because you had offered opinions about vocational rehabilitation 18 and life care issues, right? 19 A. It certainly was. 20 Q. And why don't you go ahead and pull that up, please. This 21 is the first page of the motion. Do you recognize this, sir? 22 A. Yeah. It's been some time since I've seen this, but yes. 23 Q. Specifically I want to ask you about something. Would you 24 go to page 30 of 69, and continue to the next page. Now, that 25 looks familiar. Can you -- do you recognize what that document 1463 1 is? And actually, we can go through it if you would like. 2 There's about 39 pages here. Why don't you flip through a few 3 pages, see if Mr. Duncan recognizes it. 4 A. I do. 5 Q. What is this? 6 A. This is my expert report in that case. 7 Q. Would you go to pages 53 to 55 of the document, please. 8 What's set forth in these three pages? 9 A. These are a listing of, at that time, all cases where I had 10 been retained as an expert witness, issued an expert report, and 11 then in addition the deposition testimony and trial testimony 12 columns that we saw previously in my expert report in this case. 13 Q. And so if you want to flip back, you can see I've put some 14 red arrows on the left-hand side. These were matters where I 15 didn't see a corollary -- you know, a listing for those cases in 16 your expert report in this case. I gather these are the matters 17 where you didn't testify but where you had rendered an expert 18 report; is that correct? 19 A. Correct. It either settled or otherwise been resolved 20 themselves before -- 21 Q. I beg your pardon? 22 A. They had either settled or resolved themselves before I 23 provided expert testimony. 24 Q. And if I'm correct, I counted that you provided expert 25 reports or testified in about 29 matters in the 50 months since 1464 1 you last testified in this case. Does that sound about right? 2 A. I think that number is awfully low. This was all the way 3 back in 2005 and we're in 2007 now. 4 Q. Good point. So it's more than that? 5 A. Yeah. I would think it would be significantly more than 6 that. 7 Q. So basically you issue an expert report or testify in a 8 matter about every, what, month, month and a half? 9 A. Probably average. Maybe a little bit more than that, but 10 yeah. 11 Q. Why don't you refer to page 44. Let's go back to the 12 expert report in this case. So we're back to Plaintiffs' 13 Exhibit 4284. And you're going to have to blow this one up, 14 obviously, but I'm looking at the Carlos Huerta matter. This is 15 a matter in the District Court, Clark County, Nevada. Do you 16 see that? 17 A. Yes. 18 Q. What were you retained to provide opinions about in that 19 case? 20 A. There were a couple of different opinions in that case. 21 One was quantification of the damage that -- Mr. Huerta was a 22 real estate broke r and he claimed that -- well, he actually 23 bought the rights to this lawsuit from another person, but he 24 was a real estate broker and he was suing Horizon Ridge Medical 25 Plaza for alleged breach of contract associated with selling a 1465 1 property upon which they were going to build a medical office 2 building. 3 Q. Well, you also provided opinions about whether a 4 transaction was a fraudulent conveyance, right? 5 A. Correct. That was the second of the two opinions. 6 Q. Why don't you tell us about that opinion. 7 A. This was simply an analysis of whether or not Horizon Ridge 8 Medical Plaza -- the owners of Horizon Ridge Medical Plaza made 9 a distribution to the owners at one point in time, and there was 10 a claim brought in the course of the other litigation, there was 11 a claim brought that that transfer in fact made Horizon Ridge 12 Medical Plaza insolvent. So as a follow-up report I was asked 13 to analyze whether in fact that transfer was a fraudulent 14 transfer and whether it resulted in them being insolvent. 15 Q. And what kind of expertise did you apply in making that 16 determination? 17 A. Again, basic economic finance skills, as we were looking at 18 the financial statements. As a chartered financial analyst, I 19 frequently look at financial statements to try and understand -- 20 I've testified in bankruptcy court before on the soundness of 21 plans presented to the bankruptcy judge. So here I specifically 22 was looking at whether or not that transaction rendered the 23 company insolvent. 24 Q. Can you pull up the expert opinion in that matter, please. 25 Mr. Duncan, I've had this put up on the screen now. Does this 1466 1 look familiar, sir, what's up here? 2 A. Yes. 3 Q. What is this? 4 A. This is my second expert report, the one that dealt with 5 the fraudulent transfer issue. 6 Q. Okay. And this, you stated that you concluded "Defendants' 7 actions did not wear a significant number of the badges of fraud 8 to constitute an intentionally fraudulent transfer." Badges of 9 fraud, is that a legal concept? 10 A. It is. It's articulated in the Nevada revised statutes. 11 Q. And so you were rendering another legal opinion. Is that 12 correct? 13 A. That's not correct. That was specifically a topic of a 14 motion in limine to have this opinion excluded, which was also 15 unsuccessful. The judge deemed that it was appropriate for me 16 to opine to the financial component of those badges of fraud, as 17 it related to the claim that it was an intentionally fraudulent 18 transfer. Many of those badges turn on specific issues, like 19 how much net equity does the company still have after the 20 transfer, and those were squarely within my skill set as an 21 economist and a financial expert. 22 Q. You said this was the subject of a motion in limine that 23 was also not allowed; is that correct? 24 A. Correct. 25 Q. Was the motion in limine in Karron Power ruled on? 1467 1 A. I don't recall that specifically, but as far as I'm aware, 2 I've never had a motion in limine to exclude my opinion accepted 3 by the court. 4 Q. Well, you didn't mean to suggest that the Karron Power 5 motion had been denied; is that correct? 6 A. I'm not aware one way or the other. I ultimately -- I 7 believe that case settled. I believe I testified at deposition 8 perhaps, and then the case settled. 9 Q. Let's go to page 44 of your expert report, please. 10 THE COURT: Mr. Warshawsky, are you going to bring 11 this man up on charges of unauthorized practice of law? 12 MR. WARSHAWSKY: No, I'm actually going to get to 13 the -- 14 THE COURT: Then let's get on with the statistics part 15 of this. 16 MR. WARSHAWSKY: Okay. Very good. I do have to ask 17 Mr. Duncan about one matter he testified in very briefly, 18 because it does relate to some of the opinions he's offered 19 today. 20 BY MR. WARSHAWSKY: 21 Q. The Harbor Breeze matter. That was one before the 22 California Public Utilities Commission earlier this year? 23 A. Correct. 24 Q. And your opinions were rejected in that, right? 25 A. The application was rejected. 1468 1 Q. Well, in fact, didn't the PUC find that your assumptions 2 were unrealistic? 3 A. I testified in front of the Public Utility Commission. I 4 did not review their findings at the end of the day. I 5 understand that they did allow this other party to come in and 6 begin to provide ferry service back and forth to Catalina 7 Island. But as far as what the basis of the specific 8 conclusions were, I'm not aware. 9 Q. Well, you had made an assumption that, what was it, five 10 trips a day or something like that in your analysis? 11 A. My analysis went to an estimation of how would the existing 12 ferry service -- very simple notion. The ferry service, they're 13 given a limited monopoly power to operate both during the summer 14 and the winter. They make lots of money during the summer. 15 They don't make much money during the winter because there are 16 not as many people going out to Catalina Island. 17 And my task was, in meeting with the chamber of commerce 18 and the mayor in the city of Catalina, to try to come up with an 19 estimate of, if someone else was allowed to come in and operate 20 during the summer -- sorry, I was trying to speed up for the 21 Judge. 22 THE COURT: You can't speed up fast enough for the 23 Judge. 24 BY MR. WARSHAWSKY: 25 Q. I'll take the cue there. Let's go straight to Plaintiffs' 1469 1 Exhibit 4486, then. Actually, you all have that. This is 2 Plaintiff's Exhibit 4486 that you testified about at length 3 today. Let me ask you, first of all, when you prepared -- when 4 did you prepare this chart, by the way? 5 A. We finished last night. 6 Q. When did you start working on it? 7 A. Saturday morning. 8 Q. And who worked on it? You worked on it, I gather? 9 A. I did, as well as two of my staff. 10 Q. Now, when you prepared the chart, did you study the entire 11 DCV report before reaching your conclusions there? 12 A. I read the original DCV and the DCV that was presented 13 September 30. I understand there were some additional documents 14 that were subsequent to that. I don't know that I've reviewed 15 those. 16 Q. Did you read the Alaska regional report in connection with 17 the DCV? 18 A. I believe so. 19 Q. Can you pull up DX 153A, 20. Actually, before we do that, 20 let me ask you, go back to 4486. Sorry, we're kind of making 21 this up as we go, because we did just get this ourselves. 22 Your missing months, column 7, how did you compile the 23 numbers shown under column 7 for the months of missing data? 24 A. There's a footnote there that explains that. We took the 25 Tab 2 regional reports for months prior to the IRMS conversion. 1470 1 And then for those six regions where we didn't have any 2 information, we took an average of the months that we had for 3 the regions that we did have. 4 Q. In other words, if a region didn't have data in December 5 1984, you counted it as a missing month, right? 6 A. Correct. 7 Q. And meaning that it didn't have data for the IRMS system, 8 right? 9 A. Correct. 10 Q. You assumed, did you not, that every one of those regions 11 converted to IRMS as of October 1984? 12 A. That was the implicit assumption in trying to estimate 13 whether or not there were missing months for those regions. 14 Q. And so if in fact a region converted after October 1984, 15 your assumption's erroneous, right? 16 A. It would potentially change the calculations. 17 Q. If, for example, Alaska converted around February of 1987, 18 you would have counted about, what, two and a half years of 19 missing data that wasn't really there, right? 20 A. That would be correct. 21 Q. And you don't know, for example, when the Western Region 22 converted, do you? 23 A. I believe that information is in the DCV. 24 Q. Well, if I told you between October 1985 and March 1987, 25 does that sound consistent with what you recall? 1471 1 A. Roughly. 2 Q. Okay. So to the extent your model assumed that the Western 3 Region converted in October 1984, you've overstated the number 4 of months of missing data there as well, right? 5 A. That's correct. 6 Q. And if I told you that the Pacific converted between March 7 of 1986 and November of 1986, again you've overstated the 8 Pacific. 9 A. There might be 12, 15 months of data there where we don't 10 know whether or not they had converted, or if we did find out 11 they had not yet converted, that would be included in this. 12 Q. By the way, back in the Western, if in fact Fort Yuma 13 didn't convert until 1996, you've assumed about 12 years of 14 missing data that really wouldn't be there because they weren't 15 part of IRMS until 1996, right? 16 A. That would potentially be correct. 17 Q. Potentially be correct. How would it not be correct? 18 A. You have to go back and look specifically to see whether or 19 not any of these regions had been on essentially other previous 20 electronic systems, or if this was their first conversion to 21 IRMS. Because I know that many of these actually were on 22 electronic systems well before the time we are talking about the 23 conversion to IRMS. They were on the systems all the way back 24 to the 1970s. So depending on how that exactly is being 25 accounted for on the DCV, it may or may not overstate the number 1472 1 of months of missing data. 2 Q. Why don't you pull up Exhibit 153A. Let's go to page 20. 3 Mr. Duncan, did you review this particular page when you were in 4 the process of preparing Plaintiffs' Exhibit 4486? 5 A. I believe that I have reviewed this. 6 Q. And would you read the fourth and fifth paragraphs? That's 7 actually the paragraph above the highlighting, and the 8 highlighted paragraph. You can just read that to yourself. 9 (Witness reviewing document.) 10 A. Okay. 11 Q. Did you consider that when you concluded that Alaska had 16 12 months of missing data? 13 A. I don't recall looking at that specifically as we were 14 trying to pull together what the flip side was of what 15 Ms. Herman had testified, how much they had gotten done, I don't 16 recall specifically looking at this and trying to calculate how 17 much was left to be done. 18 Q. Well, if you had looked at that, would that cause you to 19 reconsider your conclusion that there were 16 months of missing 20 data for Alaska? 21 A. Yeah. It may. Again I'd have to go back and look and see 22 if Alaska had converted to another electronic system prior to 23 this. 24 Q. Would you pull up page 33 from 153A. Did you look at this 25 page when you prepared Plaintiffs' Exhibit 4486? 1473 1 A. I don't recall looking at that specifically. 2 Q. Do you know what that means, by the way, when it says all 3 accounts in the region rolled forward? 4 A. This is what we talked about earlier today of their 5 accounts being out of balance. If one account from the ending 6 month doesn't match on the opening of the next month, then 7 you've got -- potentially have an issue that it hasn't been 8 rolled forward correctly. 9 Q. Was that an issue in this case? 10 A. It is not. It would surprise me if it was too big an issue 11 in Alaska, because they were fairly recent in coming on to some 12 of these systems. 13 Q. And had you reviewed this particular page, would that alter 14 your position about 16 months of missing data for Alaska? 15 A. Again, I think as I've already answered, I'd have to go 16 back and look and see, make sure there wasn't another system 17 they had transferred onto prior to this. 18 Q. Let's go back to 4486, please. Another assumption you 19 made, I believe -- I assume anyway in preparing this chart, is 20 that an agency or region would have transactions every single 21 month; is that correct? 22 A. Correct. 23 Q. And in fact, do you know if that's the case? Were there in 24 fact some agencies, agency offices that had months where they 25 didn't have transactions? 1474 1 A. It may be possible. To the extent that there were months 2 where agencies didn't have transactions, that should be 3 accounted for in the weighting, of going back and looking over 4 all the months we do have information. If there were instances 5 where there were not transactions, it would drag down the 6 average number of transactions per month or the average dollar 7 amount per transaction. 8 Q. But if an agency office, for example, didn't process 9 transactions in a month, you treated that as a missing month, 10 right? 11 A. That's correct. Well, let me back up. It is treated as a 12 missing month with respect to what's reported in the DCV, and 13 that was the basis upon which we did the calculation. The 14 missing month column. 15 Q. Did you read Ms. Herman's testimony in this matter? 16 A. Yes, I did. 17 Q. Do you remember reading the cross-examination when she was 18 asked about whether agencies -- whether there were any agency 19 offices that regularly processed transactions on a monthly 20 basis? 21 A. I recall there being uncertainty as to whether or not some 22 of those missing months were potentially due to them not 23 processing transactions, but that that had not yet been resolved 24 definitively. 25 Q. Well, if in fact your assumption in this chart is 1475 1 incorrect, that an agency office processed transactions every 2 month, if that's wrong, your chart again overstates the number 3 of missing months data, right? 4 A. You know, whether or not it had been processed during that 5 month, you'd have to know whether or not when, the month that it 6 did get transferred, did it -- were the transactions that were 7 processed in the missing month processed in another missing 8 month's transactions, because it may be that they lumped them 9 all the next month. If that month is missing, then the way that 10 we've done the calculation, it would average out across both 11 those months. 12 Q. Since you mentioned averaging, you used, what, average 13 dollar per transaction; is that right? I seem to remember that 14 somewhere in here. Is that right? 15 A. Yes. 16 Q. Which column was that? 17 A. 4. Average throughput per transaction. 18 Q. In fact, isn't it true that there were some transactions 19 that were very large in dollar amount? 20 A. Correct. 21 Q. And many, many small transactions? 22 A. Correct. 23 Q. Doesn't it overstate the impact in your chart to use an 24 average for the transactions? 25 A. I'd love to have the information available to me to be able 1476 1 to do the other calculation, but with the information we had, it 2 was limited to being able to do this analysis. So it depends on 3 how the weighting works out between the number of transactions. 4 It's not part of the DCV that we can tell. 5 Q. And one other thing, going back to the 1980s, do you know 6 were there in fact -- what was the practice in terms of -- do 7 you remember whether there was a practice in terms of paying 8 interest every six months or whether interest was posted on a 9 monthly basis? 10 A. I may have known that at one point in this case, but I 11 don't recall now. 12 Q. So if in fact early on, for purposes of your analysis, 13 interest was posted every six months instead of every month, 14 again your chart would overstate the impact of missing 15 transactions, right? 16 A. It may potentially. You'd have to go back and look and see 17 how they actually broke out between the months. Again, data we 18 don't have access to. 19 Q. Okay. All right. In your report, let's go to 4284, 20 please. Now, you used an expression numerous times in this 21 report, you talked about Interior's plan -- we can look at page 22 5, for example. It's in the first paragraph, the heading right 23 at the top. Just do the paragraph. 24 See right there in the first sentence you talked about -- 25 used this expression about whether something, quote, "would 1477 1 likely confuse and mislead the layperson." 2 A. Correct. 3 Q. What kind of -- what field were you using or what expertise 4 were you using in reaching those conclusions? Is that a 5 statistical judgment? 6 A. It is. As a statistician and certainly as an expert 7 witness, I think when you're providing testimony to a jury or to 8 the judge you need to take into account who that audience is. 9 And when you provide information, you need to present it in such 10 a way that it wouldn't mislead that consumer of the information. 11 Q. Okay. And are there any standards in statistics you apply 12 in reaching that conclusion? 13 A. No, but based on years of experience, you can have some 14 idea as to who your audience is and whether or not they will 15 correctly interpret and use the information you're providing, or 16 whether you need to be more clear and not use statistical 17 jargon, but you need to be clear and make sure that it doesn't 18 potentially mislead someone. 19 THE COURT: Mr. Warshawsky, for your information, and 20 I guess for the information of all counsel here, I've received 21 this witness's report in evidence, but as you may know, in jury 22 trials it is not usual to receive an expert report in evidence. 23 An expert report is handed over to the other side so they can 24 cross-examine it. But what counts is the opinions rendered in 25 open court. 1478 1 This witness has testified in open court about his opinions 2 about irrelevant sample design, sample selection problem, et 3 cetera. He hasn't said anything until you raised the subject 4 about confusing and misleading the layperson, and I'm not going 5 to take that into account. You can cross-examine it if you'd 6 like, but what's written in his report means nothing to me. 7 MR. WARSHAWSKY: I would like to use, Your Honor, with 8 your indulgence, I would like to use the executive summary in 9 his report as a basis for talking, organizing the opinions that 10 he talked about today. 11 THE COURT: You can organize it any way you want to. 12 I just want to give you a heads-up that I don't care about 13 what's written in this report unless he's also testified to it 14 in open court. 15 MR. WARSHAWSKY: Very good, Your Honor. 16 BY MR. WARSHAWSKY: 17 Q. You testified quite a bit about sampling, but in fact, I 18 believe you actually frequently stated that you lack the 19 information necessary to evaluate the sampling plan that 20 Interior's put out, right? 21 A. I think I've testified that there's information about the 22 reconciliation activity that was clearly lacking. The 23 information that we had I thought formed a reasonable basis for 24 me reaching the conclusions about the sampling plan, 25 particularly then as I was able to read the information provided 1479 1 in the rebuttal reports. It was very clear to me what the 2 intentions and what the design was originally constructed to do. 3 Q. Okay. Let's look at the first bullet point here under the 4 executive summary. You talked about missing and destroyed data, 5 drawing from a target population not being possible. You 6 testified at length about that today, right, sir? 7 A. Correct. 8 Q. And isn't it true that the mere fact that some data is 9 missing or has been destroyed, that doesn't mean that there are 10 no other copies of that information available, right? 11 A. Doesn't necessarily mean that there are not other copies. 12 It doesn't mean that there are. It simply means there have been 13 data that are destroyed or missing. 14 Q. Have you reviewed any of the testimony in this case 15 discussing the redundancy of information within Interior's 16 system? 17 A. I recall that being in your opening comments. I don't 18 recall specifically other testimony that related to 19 redundancies. You may refresh my recollection. 20 Q. Well, did you read Dr. Scheuren's testimony? 21 A. I did. 22 Q. You don't remember him talking about redundancy? 23 A. It's not jumping off the page at me. 24 Q. Okay. Is it your opinion that it's impossible to take a 25 valid sample unless you can confirm that the sampling frame 1480 1 duplicates the target population? 2 A. We went around and around on this subject years ago in my 3 deposition. But the notion -- you can take a sample from 4 anything you want to take a sample from. The notion is what can 5 you do with it when you're done. So you're asking me is it 6 impossible to take a sample? No, you can take the sample. It 7 simply limits what you can say with the information from that 8 sample. 9 Q. Well, my question for you -- and Judge Robertson wasn't in 10 our deposition years ago, and he wasn't at the 2003 hearing. My 11 question for you is is it your opinion that the only way you can 12 take a valid sample is if the sampling frame duplicates the 13 target population? 14 A. You can take a valid sample from anything you want to take 15 a sample from. The question is what can you do with that sample 16 once you're done. Can you say anything about a part of that 17 population that was not available to draw a sample from? It is 18 my opinion that no, you cannot. 19 Q. Well, why don't you pull up the Reference Manual on 20 Scientific Evidence, Second Edition 2000. Mr. Duncan, are you 21 familiar with this document? 22 A. I am. 23 Q. Would you agree that it's a learned treatise with regard to 24 the various subjects covered within that report? 25 A. I think generally speaking it is. 1481 1 Q. Okay. And if you go to page 91 of the document, please, 2 this is the reference guide on statistics. Have you seen this 3 before? 4 A. I have. 5 Q. Now, would you agree that this is a learned treatise on the 6 application of statistics? 7 A. Again, I would say generally it is. 8 Q. Go to page 240, please. I'm sorry, it's actually not that 9 page. It's page -- we're going to have to bump it forward. 10 There we go. The one that has the highlighting. Are you 11 familiar with this section? 12 A. I am. 13 Q. In fact, we talked about that four years ago, right? 14 A. Correct. 15 Q. What is the question being posed about the sampling frame 16 approximating the population? What's your understanding as an 17 expert? 18 A. It really goes right to this issue of have you constructed 19 your sample, the sampling frame, which is how you're going to 20 draw the sample, have you done it in such a way that you can 21 approximate what the population is that you would like to make a 22 statement about. 23 Q. And you see the last two sentences that I've highlighted 24 there: "Frequently, however, the target population includes 25 members who are inaccessible or who cannot be identified in 1482 1 advance. As a result, compromises are sometimes required in 2 developing the sampling frame." That's what we're talking about 3 here, right? 4 A. Correct. 5 Q. So at least according to this treatise, you can go ahead 6 and draw a sample even if there are inaccessible or missing data 7 elements, right? 8 A. We agree on this point. You can draw the sample. The 9 compromise that's talked about here is whether or not you're 10 going to be able to make any sort of statement about something 11 that was not available to be sampled from. 12 Q. Okay. And you're aware from your review -- and by the way, 13 in your expert report -- I'll tell you, I didn't go through it. 14 I think you indicated you reviewed just about everything in the 15 administrative record, right? 16 A. When I prepare my expert report, to be the state court, 17 federal court, or U.S. District Court, I prepare them in 18 compliance with the federal rules of civil procedure, and that 19 would require me to list every document that we have received, 20 also called documents considered. I don't limit it to documents 21 relied upon. So if we received a document, we listed them all, 22 whether or not we reviewed them. 23 Q. Oh. So even though in your listing of documents you 24 considered, you may not have actually reviewed documents that 25 you listed as having been considered? 1483 1 A. Correct. My understanding is that the federal rules 2 require you to list everything that has been received, whether 3 or not you reviewed it, or whether or not you've relied upon it. 4 Q. You obviously reviewed materials regarding efforts to 5 restore transactions. The DCV, you've talked about that, right? 6 A. Correct. 7 Q. Aren't those the kinds of efforts that were being discussed 8 in that treatise about attempting to take steps to fill those 9 data gaps? 10 A. Certainly. That's one of the steps that would be 11 anticipated when you have not been able to sample from a target 12 population that you want to make an inference about, you can go 13 back and see if there's some way to identify why those 14 transactions in this instance were missing, and then you can 15 draw subsamples of those to try to see whether or not they're 16 different. 17 Q. You talked about this. Why don't we go ahead and pull up 18 Plaintiff's Exhibit 4485. This is your demonstrative. Go to 19 the third page. This is also, I believe, depicted in your 20 expert report, right? 21 A. Correct. 22 Q. Now, in the second box there, it looks like the recorded 23 transactions is roughly one fourth of the total population; is 24 that correct? 25 A. Seems about right. 1484 1 Q. Do you have any factual basis for telling Judge Robertson 2 that in fact the sampling frame in this case is about one fourth 3 of the target population? 4 A. I don't think anybody knows how big that missing and 5 omitted component is. 6 Q. Okay. So as far as you're aware, the sampled population 7 and the target population may actually be fairly close in size. 8 We know they're not identical, but close? 9 A. I would have no basis, nor have I seen anything in the 10 administrative record that indicates what the relative size of 11 those two boxes is. 12 Q. So Judge Robertson, when he looks at this diagram, 13 shouldn't draw any conclusions as to the size of the, relative 14 size of those two boxes; is that correct? 15 A. No. I perhaps should put on the bottom "not necessarily to 16 scale." We don't know. 17 Q. Okay. You indicated in your direct examination that you 18 reviewed the September 30, 2005, NORC report regarding the 19 reconciliation of the national sample and the high-dollar items. 20 This was AR-438, Bates No. 40-00002. Let's try to find it maybe 21 in the administrative record. AR-438. 22 And in reaching your conclusions about missing or destroyed 23 data, the impossibility of doing the reconciliation work, you 24 considered pages 12 to 17 of the report. 25 Why don't you show Mr. Duncan 12 through 17. 1485 1 Does this look familiar? 2 A. Yes, it does. 3 Q. And on page 13 of the report, table 4, how did you evaluate 4 the fact that the table indicates that all but nine out of 2,372 5 transactions had been reconciled? 6 A. This goes right back to the notion of the high, high hurdle 7 to be considered unreconciled. 8 Q. I'm sorry? 9 A. It goes back to the high, high hurdle to be considered 10 unreconciled. So here what it's saying is if it was accounting 11 code 1 or 2, which comprises these alternative procedures, it 12 would be considered -- it would be listed in the reconciled 13 section. But because we don't know what those alternative 14 procedures are, and they may or may not be directly reconciled 15 back to a supporting document but rather be subject to some 16 judgment of the accountants, the notion of reconciled here, as 17 I've testified to, would be meaningless to an individual account 18 holder. 19 Q. Well, you're a CFA but you're not a CPA, right? 20 A. That's correct. 21 Q. And so you're not here today to offer opinions about what 22 would be adequate reconciliation procedures, right? 23 A. That's correct. My understanding is Mr. Pallais will be 24 doing that. 25 Q. You talked about not knowing what alternative procedures 1486 1 were, right? 2 A. Correct. 3 Q. So you don't know if that's an auditor's concept? 4 A. I've reviewed the ASM to see whether or not it's something 5 they laid out in the ASM as an auditor's concept. I didn't see 6 anything there that made it clear what this notion of the 7 alternative procedure was. 8 Q. Okay. But speaking more generally, you don't know whether 9 auditors routinely talk about utilizing alternative procedures 10 in the course of doing audit work; is that correct? 11 A. I've encountered the term before in working with 12 PricewaterhouseCoopers. 13 Q. When you saw that term utilized in connection with the work 14 here, did it occur to you that "alternative procedures" was an 15 auditor's term of art? 16 A. I'm generally familiar with the term as it's been used by 17 accountants when I did work in helping design statistical 18 sampling when I was at PricewaterhouseCoopers, so I've heard 19 "alternative procedures" used in different contexts among 20 accountants. 21 Q. So basically you rejected the conclusion in table 4 about 22 reconciling all but nine of the transactions because of your 23 understanding of how errors were determined; is that correct? 24 A. Well, you're mixing two different concepts. You talked 25 about reconciled and you talked about errors. Reconciled is the 1487 1 first step. The second step is whether or not there's an error. 2 Q. All right. You rejected the conclusion that all but nine 3 transactions had been reconciled because of your understanding 4 of the standard required to reconcile a transaction; is that 5 correct? 6 A. I don't know that I rejected or discounted the notion. It 7 was reported here that it was reconciled. It was just my 8 opinion that given the hurdle associated with not being able to 9 be reconciled, that this doesn't hold any particular meaning to 10 the individual account holder. 11 Q. Okay. As a statistician, though, if it's reported to you 12 that all but nine out of almost 2,400 transactions was 13 reconciled, in your mind you would still consider there to be a 14 problem with missing data? 15 A. Well, this goes to how the error rates and the 16 reconciliation process is determined, and statisticians, as with 17 your clients, Dr. Scheuren and Dr. Hinkins, had input, they had 18 involvement in the discussion of what was reconciled and what 19 wasn't. So it matters because it then -- how one uses the 20 result of that reconciliation activity is premised in part, in 21 large part upon what the reconciliation means. 22 So as a statistician, I would view with great skepticism 23 the information that all but nine were reconciled. At a minimum 24 I would like to know how many were reconciled with a directly 25 supported document, how many were reconciled by this 1488 1 recomputation method, and how did that work. How many were 2 reconciled by third-party documentation. How many of these were 3 reconciled with the alternative procedures, and how was that 4 alternative procedure actually done. 5 Then I would be able to take this information and make some 6 kind of meaningful extrapolation up to the rest of the 7 population from which I drew the sample. 8 Q. And in the course of your rendering your opinions rejecting 9 these findings, you never reviewed any of the materials used by 10 the accountants to reconcile transactions, did you? 11 A. No, that's not correct. I've testified repeatedly I looked 12 at the accounting standard manual, which is the -- all the 13 different versions we were provided, there were at least two or 14 three we looked at because it's changed over time, but we looked 15 at the accounting standard manuals, and we looked at all the 16 information we did have, which was the Eastern Region study, but 17 we were not provided as part of the administrative record with 18 any of the actual source documentation where you specifically 19 tied one transaction back to another. 20 Q. And that's what I meant by my question. You never looked 21 at any of the stuff the auditors or the accountants actually 22 looked at in concluding that a transaction was reconciled, 23 right? 24 A. I looked for it and I didn't find it. 25 Q. Just to move it along -- oh, I'm getting a time-out sign. 1489 1 THE COURT: We'll take a 10-minute break. 2 (Recess from 3:42 p.m. to 3:56 p.m.) 3 MR. WARSHAWSKY: Your Honor, just from a planning 4 point of view, counsel were talking over the break. Does the 5 Court have a 4:30 matter, or is that gone? 6 THE COURT: We do but they're going to wait. 7 MR. WARSHAWSKY: Excellent. Let's see if we can get 8 through Mr. Duncan, then. Maybe you'll get your flight after 9 all. 10 THE WITNESS: I was just telling His Honor thank you. 11 BY MR. WARSHAWSKY: 12 Q. Just jumping back to one subject, you read Ms. Herman's 13 testimony? 14 A. I did. 15 Q. You recall her testimony about the Plaintiffs being 16 provided copies of the IRMS data in 1997? 17 A. I do recall that. 18 Q. In the course of your work, did you ever consider analyzing 19 the data that was provided to the Plaintiffs? 20 A. No, I have not. 21 Q. Now, let's go back to AR-438. Page 16, table 7. Do you 22 remember I was asking you before the break about the debit 23 transactions. I guess to shortcut it, credit transactions I 24 gather you would draw a comparable conclusion with regard to the 25 ability to reconcile the items in the national sample, missing 1490 1 data? 2 A. That's correct. Although here there are more differences 3 noted, but the same concept applies. 4 Q. In this case -- by the way, you remember what Dr. Scheuren, 5 what NORC did for those instances where they couldn't find 6 support for a transaction? 7 A. Well, you have to be more specific. What do you mean? In 8 which calculation? 9 Q. I will. Why don't you go back -- let's jump back, I'm 10 sorry, in the debit transactions. Okay. It's in the inferences 11 for debit transactions. Do you remember reading this section, 12 inferences for debit transactions? 13 A. Yes. 14 Q. Does that refresh your recollection as to how Dr. Scheuren 15 and the NORC team treated the debit transactions where they 16 weren't able to reconcile them? 17 A. Again, there's still the same two questions: How was it 18 treated from the attribute standpoint and how it was treated 19 from the variable standpoint. From the attribute standpoint it 20 was treated as an error, for the calculation of it being a 21 yes/no, was it an error. But for purposes of calculating it for 22 the variable, the dollar amount of the error, since it was 23 unreconciled, there was no information available, and the NORC 24 report assumed that that dollar difference was zero. 25 So it's some of both. It's an assumption about what the 1491 1 error rate is, it was treated like an error. For the dollar 2 difference, it was not. 3 Q. And what's the basis for your conclusion that that's how 4 they treated it for the variable analysis? 5 A. I'd have to go back and look at the documents. It's in 6 some of the detailed NORC reports, where they say there's a 7 couple of different ways we can deal with this when we don't 8 have information on the reconciliation. We could treat it as 9 the average dollar error for the reconciled transactions, where 10 there was an error, we could take that average dollar amount, or 11 we could treat it as a zero. And they've effectively treated it 12 as a zero. 13 Q. Are you talking about some of those draft documents and 14 some of those documents prepared, say like in the 2002, 2003 15 time frame? 16 A. I'd have to go back and look specifically where that was 17 at. 18 Q. You don't sitting here today remember any analysis done in 19 2005 that treated the unreconciled items as being not errors for 20 purposes of the variable analysis, do you? 21 A. Again, I'd have to go back and look at what date that 22 document was. 23 Q. And similarly, we can go to it -- by the way, Dr. Scheuren 24 says that, in the final paragraph up here, that the error rates 25 that he calculated, that those are based on a very conservative 1492 1 estimate, treating them as unreconciled, right? 2 A. For purposes of the attribute piece, yes. 3 Q. It's about as conservative as you could be, right? 4 A. No. I think Dr. Scheuren or Dr. Hinkins appropriately 5 indicated it would be an unsound statistical practice to do 6 otherwise. In her rebuttal report she specifically opined that. 7 I wouldn't describe it as being conservative. It's the right 8 way to do it. 9 Q. Thank you. I'll take that. Similarly for the credit 10 transactions, do you remember how the NORC team and Dr. Scheuren 11 treated unreconciled credit transactions? 12 A. My recollection is that there weren't any unreconciled 13 credit transactions. 14 Q. Why don't we jump ahead. Let's get to the inference for 15 credit transactions. This actually spills over to two pages. 16 There is this page right here, we're looking at this, what, 16, 17 the last paragraph on 16, where he sets out his percentages and 18 error rates and all that, and then spilling over to the next 19 page, go to the top of 17, you see the first full paragraph. 20 Does that refresh your recollection as to how the unreconciled 21 transactions were calculated? 22 A. No. I'd have to go back and look at what component of this 23 is unreconciled. This is just dealing strictly with what the 24 error rate is, the 4 percent error rate. You'd have to go back 25 and look and see what proportion of those were unreconciled. 1493 1 These are errors. 2 Q. I'm sorry, I'm looking at the next paragraph where he 3 actually quantified the $86 million, the 42 million. Do you see 4 that? 5 A. Yes, I do. 6 Q. Does that refresh your recollection as to how the NORC 7 analysis treated unreconciled credit transactions? 8 A. No, it doesn't tell you anything about what -- if there was 9 a difference, what that difference was. And on an unreconciled 10 transaction, it would be unclear to me what basis they would 11 have for what that difference was. It's not been reconciled. 12 Q. Okay. Let's go back to your expert report, second bullet 13 point on page 5, please. Why don't you blow that up. This is 14 the one, do you see that, about the sampled population is 15 different from the target population? 16 A. Yes. 17 Q. And you say that the plan to sample the target population 18 is conjecture, not statistical inference. Does your analysis 19 also take into consideration the work that the plan describes to 20 be completed with regard to things such as the DCV, the 21 land-to-dollars work, posting tests? 22 A. No, this specifically looks at the sampled population, 23 which was the subject of the sampling that was done, cannot be 24 inferred beyond the target population from which it was drawn. 25 Q. So you're not telling Judge Robertson that if Interior then 1494 1 goes ahead and performs the remaining work with regard to 2 posting transactions, land to dollars, the DCV work, that it's 3 not possible to draw valid statistical inferences about the 4 entire population here, are you? 5 A. Well, you would need to make sure that we don't have 6 missing and omitted transactions. And based on my review of the 7 documents, that is a hurdle that will not be cleared. As Arthur 8 Andersen opined, as the GAO opined, doing that kind of 9 reconciliation would not be possible given the paucity of 10 documents. 11 Q. Well, I understand that's your assumption. That's what 12 you're working from. My question for you, sir, is if Interior 13 does go ahead and performs the remaining work to be done which 14 is described in the plan -- and this is in the administrative 15 record if you want to look at it, we could pull that up, at 16 pages 17, 18 AR5-66. Do you want us to do that? 17 A. We certainly could, but what you would need to see is the 18 results of those, just as we've now, in 2007, have seen the 19 results of the sampling, you'd need to see the results of those 20 activities to opine whether or not it can be extrapolated to the 21 target population of all trust beneficiaries for all funds in 22 transactions and account balances. 23 Q. And at this point in time you are just not in a position to 24 tell Judge Robertson anything about those results because they 25 haven't happened yet, right? 1495 1 A. Well, I'm in a position to tell him that, in the first 2 instance, you're not going to be able to get to account 3 balances. It doesn't matter how much more sampling you do, 4 you're sampling transactions, so you're not going to be able to 5 get to -- when I read the 2007 Plan it's clear to me, it says 6 there would be accuracy and completeness with respect to the 7 account transaction listings and the account balances as of 8 12/31/2000. So any extrapolation out to that population at the 9 end of the day is not going to be supported by what I've read in 10 the 2007 Plan. 11 Q. Okay. I think we're talking about a different point now, 12 and we'll get to the account balances and the transactions 13 issues that you testified about. But let's move to the third 14 bullet point here. You talked about estimating litigation 15 exposure, and I'm going to be real quick on this one. You've 16 read Dr. Hinkins' rebuttal to your opinion on that, right? 17 A. I have. 18 Q. And she says you're just flat out wrong, right? 19 A. Well, she's of the opinion that it was estimated to do what 20 it did, which is to make an extrapolation to the sliver of the 21 electronic ledger from which they sampled. So we're obviously 22 in disagreement on this point as to what it was designed for, 23 but the notion of what the dollar exposure is was not only 24 testified to by Dr. Scheuren but was also specifically in the 25 NORC documentation of the LSA project. It's $86 million in 1496 1 exposure on credits. $8 million in exposure on debits. That's 2 exactly the language, so -- 3 Q. I think it was 4 on debits, right? 4 A. 4 million on debits. I stand corrected. So those issues, 5 the concept is unchanged that there's this notion of what the 6 exposure is. The exercise that was done does identify the 7 exposure during the electronic ledger era in total, but it's not 8 going to give information to the account holders about their 9 account balances. 10 Q. Well, specifically, I'm talking about your analysis, and 11 it's in 4.3.3.1, which you reference in this bullet point. We 12 can go there if you need. But you talked about the netting of 13 transactions, netting of overpayments and underpayments, right? 14 A. Correct. As one specific illustration of this being a 15 litigation exposure exercise, estimating the numbers in total, 16 rather than an analysis, as I've said here, of the histories -- 17 transaction histories and account balances. 18 Q. And in Dr. Hinkins' rebuttal to you she was pretty adamant 19 that was wrong, your assumption, right? 20 A. You mean the concept of netting. 21 Q. Right. 22 A. She was adamant that NORC had not done netting, but as I 23 read her opinion, she didn't say that the 2007 Plan had not done 24 netting. 25 Q. Okay. Well, if your assumption about the netting of 1497 1 overpayments and underpayments is incorrect, you'd agree with me 2 your third bullet point is also incorrect here, right? 3 A. No, but that's only one piece of it. It's also what we've 4 talked about, there not being an ability to talk about the 5 account balance. 6 Q. Now, the fourth bullet point, and you started talking about 7 this just a couple of answers ago, about the notion of sampling 8 transactions instead of sampling account balances. Let me ask 9 if, from a basic point of view, as a financial person, you 10 understand of course that an account balance is the result of 11 the transactions, right? I mean, the sum of transactions going 12 in, reductions for transactions going out, right? 13 A. You can get there if you know the opening balance, yes. 14 Then the transaction information will help you get to a closing 15 balance. 16 Q. Okay. And in fact, isn't it true -- maybe you don't know 17 this, I don't know, but from your dealings with accountants, 18 isn't it true that one of the things that auditors do is 19 assess -- is look at transactions to come to a conclusion about 20 how reliable the business systems are that are used to generate 21 reports such as account statements? 22 A. Well, certainly when you're looking to assess the integrity 23 of a system, one of the tools that you can use is to measure 24 transactions. That's certainly one tool. 25 Q. Okay. And if, for example, somebody's account opens as a 1498 1 result of a probate transaction, I mean that's a transaction 2 that creates the opening balance in the account, right? 3 A. That would do that, yes. 4 Q. Okay. 5 A. Potentially. I guess if they didn't have another account, 6 then yes, a probate transaction could create a separate account. 7 Q. Okay. So that's one way that you could be looking at the 8 accuracy of what the opening balance is, you look at the 9 transaction that establishes the opening balance, right? 10 A. That would be one way. 11 Q. Okay. Now, in your report, you stated, and you testified 12 about this earlier, that Interior was aware of the issue about 13 sampling accounts, and you referred to this November 24, 2002, 14 memorandum. Do you recall that? 15 A. Is that the OHTA memorandum? 16 Q. It is. Why don't we go ahead and pull that up. This is 17 administrative record 264, Bates numbers 11-5. And you 18 testified about this earlier today, right? 19 A. Yes, I did. 20 Q. And specifically, you were referring to page 4. Let's go 21 ahead to page 4, and blow up -- the second paragraph is what 22 you're talking about. Why don't we get both paragraphs, the one 23 above as well. There we go. 24 Specifically, you focused on this statement about the 25 account error rate could explode up to 30 percent, right? 1499 1 A. Correct. 2 Q. And in your analysis, did you also consider the preceding 3 paragraph, where the author of this draft -- and by the way, 4 this was a draft document, wasn't it? 5 A. I believe so. 6 Q. -- was noting the cost of doing a full reconciliation of 7 accounts? 8 A. Correct. 9 Q. In fact -- and you've talked about the paragraph 19 10 exercise a little bit, right? 11 A. I did. 12 Q. And you know that was an extremely expensive process. 13 A. Correct. 14 Q. To select an account and reconcile every single transaction 15 in it, right? 16 A. Correct. 17 Q. Isn't that the context of the statement here, that it 18 wasn't a focus on the error rate caused by accounts, sampling, 19 it was the cost of doing account reconciliations, 100 percent 20 verification? 21 A. You're talking about two different concepts. You're 22 talking about a hundred percent reconciliation on all accounts, 23 versus you're talking about taking a sample of an account and do 24 an analysis of every transaction in that account. The result 25 that if -- because sampling transactions, even with a very low 1500 1 error rate, will result in a relatively high error rate, as far 2 as an assurance standpoint on account balances, I think that is 3 portrayed here accurately, and as a separate issue from the 4 overall cost. 5 It's just another weakness they've issued, is if we do this 6 transaction-by-transaction sampling, everybody needs to be on 7 notice that we could only be able to say very limited things, or 8 I guess as it turns out nothing, about account balances. 9 THE COURT: Mr. Warshawsky, you yourself are the one 10 who has interposed a number of "speaks for itself" objections. 11 You also want to downplay this exhibit as only a draft. Why do 12 you need to quarrel with this witness about his interpretation 13 of somebody else's document that's only a draft and it speaks 14 for itself? 15 MR. WARSHAWSKY: Your Honor, I won't -- first of all, 16 I'm going to try not to quarrel with Mr. Duncan. Mr. Duncan has 17 focused on one specific statement in there and attributed a 18 motive to the Interior Department based on his reading of that 19 sentence, and I think it's simply important for the Court to 20 appreciate the full -- the context of that statement. 21 MR. GINGOLD: Objection, Your Honor. He did not 22 attribute a motive to -- 23 THE COURT: I didn't hear him attribute motive either, 24 but go ahead, Mr. Warshawsky. 25 MR. WARSHAWSKY: Well, Your Honor, it actually was in 1501 1 his report, and I think he has -- but we can move on. 2 MR. GINGOLD: Your Honor, you instructed us to deal 3 with opinions as stated in this court. I object on any further 4 discussion -- 5 THE COURT: It's gone over my head, counsel. Both of 6 you. Proceed, Mr. Warshawsky. 7 MR. WARSHAWSKY: Okay. 8 BY MR. WARSHAWSKY: 9 Q. The one thing I did want to ask you is the sentence that 10 follows what you referred to in your testimony, where the author 11 of this document wrote, "Clearly the numbers that should be 12 estimated is the amount of the dollars in error, not the 13 percentage of accounts with at least one error." Did you 14 consider that one as well in your analysis? 15 A. I did. That goes to the notion that what they're trying to 16 estimate is whether or not the transactions were correct, 17 whether or not the account histories as they were recorded, were 18 those correct. But it ignores the notion that, what my 19 understanding of what the Court has ordered, is the account 20 balances are required as part of that. 21 Q. If you simply look at the number of accounts that are in 22 error, or look at an account error rate, isn't that essentially 23 an attribute analysis? 24 A. It is. 25 Q. Okay. And you've criticized Interior because you believe 1502 1 they're employing attribute sampling in this, right? 2 A. I've only criticized to the extent that it won't accomplish 3 the end objective. What I'm opining here is you need get to the 4 dollar difference on the account balance. Not an attribute, 5 yes/no, is the account balance correct? What is the dollar 6 difference on the account balance. If I were an individual 7 account holder, that's what I would want to know, is my balance 8 right or not. 9 Q. So you'd advocate using a variable analysis, right? 10 A. I certainly think you could do both attribute and variable, 11 but in the end, as an account holder, I'm not really going to 12 appreciate it much if you just tell me your account balance is 13 wrong. I would like to know by how much. 14 Q. That's accomplished through a variable analysis, right? 15 A. It could in part. It could be accomplished by an attribute 16 analysis first and then a follow-up with a variable sample. 17 Q. Okay. Let's go back to your expert report. Again the 18 executive summary, page 5. Specifically, you criticized in 19 the -- what we have on the screen here now, it looks like it's 20 the fourth bullet point, but this is actually the fifth on the 21 page, the one about the statistical sampling plan -- I'm sorry, 22 the statistical sampling design employed attribute sampling. 23 A. Correct. 24 Q. In fact, having looked at things such as the September 30, 25 2005, NORC report that we've gone through just a few moments 1503 1 ago, isn't it true that Interior utilized both attribute and 2 variable sampling in this? 3 A. I didn't have the advantage of having Dr. Hinkins' opinions 4 before I drafted this report, but it's clear that they are 5 taking this the next step, which was hypothesized back in '03 6 that that might happen but not articulated how. It is now clear 7 that they are taking that next step of looking at the variable 8 portion as well. 9 Q. Okay. So that's been clarified to you. 10 A. Yes. 11 Q. Okay. Next bullet point you talked about, and you did 12 spend a lot of time talking about this in your direct. I want 13 to spend a few moments at least talking about it as well. The 14 definition of a deviation or an error. And for that, let's go 15 to page 24 of your report, please. Actually, why don't we move 16 ahead to page 27. And up at the top, you see up here -- let's 17 blow that up, please. 18 You went through on pages 27, 28 of your report your 19 analysis of the various types of transactions which you 20 concluded would not be considered errors; is that correct? 21 A. Correct. Based on my read of the administrative record. 22 Q. Well, now specifically your report seems to be citing the 23 definition of difference in error and the definition of 24 difference rate and error rate in the plan, and then you stated: 25 "Transactions not considered to be deviations according to the 1504 1 plan definitions." So you really considered more than just the 2 definitions in reaching these conclusions? 3 A. You mean in reading the entire plan? 4 Q. In reaching your conclusions below about transactions which 5 you said would not be considered deviations. 6 A. Well, what I'm saying here is that according to the 2007 7 Plan, according to my read of the plan, these notions would not 8 be considered as deviations. 9 Q. Okay. So your analysis was more than just definitions, 10 right? 11 A. Correct. 12 Q. Okay. You reviewed the accounting standards manual? 13 A. Yes. 14 Q. Okay. Did you consider how the accounting standards manual 15 described these types of errors in your analysis? 16 A. Generally, yes. 17 Q. Generally. I mean, you didn't think that that was the real 18 authority to go to to determine whether or not an item would be 19 considered an error? 20 A. Well, it first of all varies by different types of 21 transactions, it varies by different types of credits, debits, 22 whether or not they're oil and gas collections, whether or not 23 they're check disbursements. So what I tried to do was distill 24 down the essence of this notion of what's considered a deviation 25 and what's not, and it interweaves both the notion of 1505 1 reconciliation activities, part of which are then included in 2 the ASM manual, but also part of the extrapolation that's done 3 by NORC, where for instance these de minimis rules are applied 4 as to whether or not there's actually an error. 5 The accounting would catch a lot of these. Like with the 6 one we looked at, the only one we've been able to look at, the 7 Deloitte & Touche study showed us where it was off. They 8 captured that in the accounting. The de minimis rule was 9 applied later on in the extrapolation process. 10 Q. Okay. You didn't talk about any of that in your report 11 here, right? 12 A. That's correct. I testified to that content to the Judge. 13 Q. Well, let's see now. You say transactions not considered 14 to be deviations, and the first item you said recorded 15 transactions largely consistent with supporting documents that 16 are found. Are you telling Judge Robertson that a transaction 17 that's supported by -- that's largely consistent with supporting 18 documentations, that should be considered a deviation? 19 A. Yeah. If it's inconsistent -- I mean there's this notion 20 of, under accounting code 1, there's this notion of it could be 21 directly supported by a document, but it could also just be kind 22 of roughly consistent. And again, just for discussion purposes 23 here, if it's not exactly tied to a source document, I think 24 that's information, particularly as a statistician that's going 25 to try and extrapolate from it, I would need to understand what 1506 1 proportion of it exactly is only largely consistent with 2 supporting documents. 3 Q. Okay. So to understand your opinion we need to know what 4 you mean by the word "largely"? 5 A. Well, yeah. It either is consistent or is not consistent. 6 But the 2007 Plan, if it's largely consistent -- in other words, 7 if there is, by recomputing it or by looking at third-party 8 documents we're able to say it looks right, that's not 9 considered a deviation. 10 Q. Is the word "largely," is that your word or does that 11 appear in something like the accounting standards manual? 12 A. I don't recall reading that specifically in the ASM. 13 Q. Okay. Next item you listed, a group of missing, erroneous 14 or omitted transactions. With respect to the first two bullet 15 points, those that were never entered into the transaction 16 ledgers and those that were removed from the transaction 17 ledgers, wouldn't those types of transactions be addressed 18 through this work that we've been talking about, the DCV, 19 land-to-dollars testing, posting work? 20 A. They could potentially be included in that universe. 21 Q. All right. Transactions that were intentionally altered, 22 accidentally altered or erroneously entered. If that happened, 23 wouldn't those types of transactions be inconsistent with 24 supporting documentation? 25 A. Not necessarily. Again, I don't know what the alternative 1507 1 procedure that would be used to substantiate it, whether or not 2 that would detect something that was intentionally altered or 3 not. 4 Q. What do you mean by an intentionally altered transaction? 5 A. Well, it could be something that was, as Ms. Redthunder was 6 testifying to this morning, where a person was taking funds from 7 a person that's disabled or otherwise that they're watching 8 over, if they're inside of the BIA, they could go back and make 9 a transfer and intentionally alter the documentation associated 10 with that transfer. 11 Q. But ultimately if the accountant selected that transaction 12 and reviewed supporting documentation, wouldn't that reveal 13 whether that transaction had been altered? 14 A. May or may not. It depends again -- in my experience in 15 dealing with trying to identify and detect fraud in 16 organizations, those are exactly the kind of transactions where 17 you would anticipate that the supporting documentation would 18 both be altered, and so that it would support what had been 19 done. Those are exactly the ones that are high risk. 20 Q. Okay. Cases of fraud are pretty unusual, though, aren't 21 they? 22 A. Not in my experience. 23 Q. Okay. Well, let me ask you about the last bullet point. 24 Transactions that never should have occurred. I'm sorry, I'm 25 not sure what that means. What did you mean by that one? 1508 1 A. Well, using an example that I used with Mr. Gingold this 2 morning, if there was a collection that happened at Treasury, 3 there was a million dollars that was collected, if those funds 4 were posted over and it was a transaction, we should have never 5 received any money or we should have received money but didn't 6 get any, there could be a transaction that never should have 7 occurred the way that it did, that's not going to be captured 8 here as a deviation. 9 Q. But if you reviewed supporting documentation, you wouldn't 10 be able to tell whether it was a transaction that should not 11 have occurred? 12 A. First of all, you're assuming they are reviewing these 13 documents, and they're not. Under this example, they're not 14 going back over to Treasury to try and understand whether or not 15 the allocation to the allottees occurred correctly. If you're 16 asking if they were doing that step, should they detect it? 17 They probably would be able to detect something like that, which 18 is erroneous allocation. 19 Q. Okay. Last item, you talked about recorded transactions 20 where no supporting documentation could be found. Aren't those 21 the unreconciled transactions that NORC treated as errors in 22 their analysis? 23 A. No. There's lots of pieces of the reconciliation that 24 don't have supporting documents. I mean, category 1, the top 25 tier reconciled transaction, part of that have directly 1509 1 supporting documents, but part of it does not. It's recomputed, 2 it's third-party documents. So then when you drop down to 3 category 2, you know it's not something that has a directly 4 supporting document because it's already dropped all the way 5 down to category 2. 6 So these are the kinds of things that you would anticipate, 7 given it's not required to have a directly supporting document 8 in order to do the reconciliation. 9 Q. But you've already told us you don't know what's required 10 for an item to be reconciled with category 2. Right? 11 A. That's exactly what I testified to, but I know what it's 12 not, and it's not a directly supporting document or it would be 13 up in category 1. 14 Q. Okay. So what you're referring to, you're not really 15 referring to no supporting documentation, you're talking about 16 transactions that don't have what you call directly supporting 17 documentation; is that right? 18 A. Right. It's the notion of an alternative procedure. It's 19 not a supporting document. It's an alternative procedure. 20 Q. Okay. Let's go back to page 5 of your report, please. 21 Moving along. We're down to the seventh bullet point. And this 22 is the one that talks about the plan does not constitute a 23 sufficient basis for making extrapolations beyond the population 24 of recorded transactions in the electronic ledger era, subject 25 to DOI date restrictions. 1510 1 You're talking about, to cut through it, in that instance 2 you're really talking about issues like probate, direct pay, the 3 inclusion of accounts as of 10/25/1994; is that right? 4 A. Correct. 5 Q. And these are all legal issues, right? 6 A. Correct. I'm just informing the Court -- as a matter of 7 fact, that paragraph could be taken out of my report and put in 8 Dr. Hinkins' report. She has essentially the same opinion. You 9 can't extrapolate beyond the electronic ledger era from which 10 the sample was drawn. 11 Q. In other words, you're not saying there's a sampling issue 12 there; you're saying they haven't -- that the target population 13 doesn't include items that have been excluded because of legal 14 arguments. 15 A. Correct. The consumer of this report, my understanding is 16 it would be the Court, and I want to make it clear to the Court 17 that what was done will not go beyond those date restrictions. 18 Q. Okay. The final item on that is the famous meta-analysis, 19 and I gather now, based on what you've heard from -- what you've 20 read I should say from Dr. Scheuren and Dr. Hinkins and what you 21 said this morning, you recognize that meta-analysis is not a 22 central -- does not play a central role in the proposed plan for 23 the paper ledger era. 24 A. That's my understanding. 25 Q. Your Honor, if I may have just a minute. 1511 1 Let me ask one last thing. You did talk about your 2 criticism of the proposal to conduct a sample of the paper 3 ledger era to determine whether the populations were essentially 4 the same. 5 A. Correct. 6 Q. Okay. And you're familiar with the term initial hypothesis 7 testing? 8 A. Yes. 9 Q. And so you understand that that's simply a first step that 10 Interior's proposing to do. 11 A. That's my understanding. 12 Q. Based on your assumptions about the paper ledger era, it's 13 fair to say you expect that initial hypothesis test to be 14 rejected? 15 A. Again, it goes back to the definition of the error. If 16 what you're looking for is an error and you're unlikely to find 17 it, not finding it in the paper ledger era, and that happens to 18 be the same as the electronic ledger era, I would anticipate 19 that if you define the error high enough, you won't see it in 20 either, and you could try to reach the conclusion that they're 21 the same. 22 But it would be my expectation that if you were testing it 23 to talk about legitimately whether or not there's a difference 24 in the supporting documents, you would find a substantial 25 difference between electronic and paper ledger eras. 1512 1 Q. Okay. Well, let's step back from whether you believe 2 Interior is acting legitimately, whether you believe they're 3 setting the standards too high for an error. I ask you to 4 assume that the way errors are being defined are proper and 5 consistent with the standards that accountants would employ in 6 this situation. Based on what you understand, your beliefs 7 about the conditions of paper records, you expect the initial 8 hypothesis to be rejected, right? 9 A. I would. 10 Q. Okay. And in that case, do you remember what Interior's 11 proposed to do if the initial hypothesis is rejected? 12 A. Back to the drawing board and draw bigger samples. 13 Q. Thank you. 14 MR. WARSHAWSKY: Your Honor, I have no more cross. 15 Thank you, Mr. Duncan. 16 THE WITNESS: You're welcome. 17 MR. GINGOLD: A brief redirect, Your Honor. 18 REDIRECT EXAMINATION 19 BY MR. GINGOLD: 20 Q. I'd like to call up Bates No. 152-4 at 17, which relates to 21 Mr. Warshawsky's discussion of Plaintiffs' 4486, which is the 22 estimated missing transactions throughput from October 1, 1984. 23 As you can see, Mr. Duncan, this is the FTI report. And I'd 24 like to go to 17. Have you seen this before? 25 A. Yes, I have. 1513 1 Q. Isn't this tab B-2 you're referring to in footnote 7 of 2 Plaintiffs' Exhibit 4486? 3 A. Yes. This is the table. 4 Q. And if you can focus in a little bit, you're looking at -- 5 I'm not looking at that paragraph, excuse me. You're looking 6 at -- if we can just have the upper part of this a little bit so 7 we can have some clarity. I'm sorry. 8 Okay. We're looking at control tests and we're looking at 9 missing months, aren't we? Does this, part of it, provide that? 10 A. Yes. 11 Q. Okay. So you're dealing with missing months, correct? 12 A. Correct. 13 Q. And footnote 7 says some of -- footnote 7 in Plaintiff's 14 Exhibit 4486 states the sum of tab B-2 regional reports for 15 months prior to IRMS conversion and missing months after IRMS 16 conversion, correct? 17 A. Correct. 18 Q. And you're stating this as an estimate, correct? 19 A. It is. 20 Q. Based on the missing months identified in the DCV, correct? 21 A. It is based on exactly that. 22 Q. And one of the questions that was asked by Mr. Warshawsky 23 is whether or not you properly considered the missing months if 24 all the agencies or regional offices didn't start as of your 25 October 1, 1984 date. Do you recall that? 1514 1 A. Yes. 2 Q. Now, you did not consider, for example, and let's take 3 Rocky Mountain Region in the 4486. You didn't consider that if 4 Rocky Mountain Region actually went on IRMS in 1976, you didn't 5 consider the near decade of purged documents on a monthly basis, 6 did you? 7 A. I did not. I simply started with what the DCV identified 8 as the missing months. 9 Q. So when you're looking at this number, when you're looking 10 at the total number of missing months data, you're dealing with 11 it from still a narrow time period, correct? 12 A. Correct. 13 Q. You're not dealing with IRMS once it actually became 14 implemented in the various agencies or regional offices, 15 correct? 16 A. That's correct. 17 Q. So your estimate is what it is, for a period of time, 18 excluding nearly a decade in some respect of missing data, 19 correct? 20 A. At least in that one agency. It could be more or less in 21 other agencies, but it's simply limited to the missing months 22 identified by the DCV. 23 Q. And the DCV is not complete, is it? 24 A. No, it is not. 25 Q. And the land-to-dollar testing is not complete, is it? 1515 1 A. Not to my knowledge. 2 Q. And you were asked specifically questions about the 3 Reference Manual For Scientific Evidence, weren't you? 4 A. Yes, I was. 5 Q. One of the -- I'm not going to put this up, but on page 90 6 of the reference manual, there's a statement regarding when 7 inferences can be made, correct? 8 A. I don't recall page 90 specifically. It's a large manual. 9 MR. WARSHAWSKY: Your Honor, we should pull it up. 10 MR. GINGOLD: This should be a short question, 11 Your Honor. 12 BY MR. GINGOLD: 13 Q. Is this a fair statement made in the manual: Inferences 14 from part to the whole are justified only when the sample is 15 representative? 16 A. That would be the kind of statement in the scientific 17 manual, yes. 18 Q. And one of your concerns with regard to your opinion is 19 that the sample is not representative of the target population, 20 correct? 21 A. That's been my concern since I first started looking at 22 this. 23 Q. Therefore, your opinion remains the same; if in fact the 24 sample is taken for the purpose of establishing accurate account 25 balances, for all past and present individual Indian trust 1516 1 beneficiaries, it is not representative, is it? 2 A. The sample cannot be used to make a statement about that 3 population. 4 Q. And Mr. Duncan, none of your opinions have changed as a 5 result of the cross-examination by Mr. Warshawsky, have they? 6 A. No, they have not. 7 MR. GINGOLD: Thank you. 8 THE COURT: I think you're excused, Mr. Duncan. Thank 9 you. 10 THE WITNESS: Thank you, Your Honor. 11 (The witness steps down.) 12 THE COURT: Do you have a 25-minute witness this 13 afternoon, or do you all want to go home? 14 MR. DORRIS: Our next witness is Mr. Homan. We can't 15 complete him in 25 minutes, but we can certainly start him if 16 that's what you'd prefer to do. 17 THE COURT: How long is Mr. Homan's direct? 18 MR. DORRIS: I would say it's about an hour and 45 19 minutes. 20 THE COURT: Let's get started. 21 MR. DORRIS: We call Paul Homan. 22 THE COURT: All right. 23 MR. DORRIS: Your Honor, we're going to cover several 24 areas with Mr. Homan. He was the first special trustee from 25 September of 1995 to January of 1999 pursuant to the 1994 Act, 1517 1 after he had been with the Office of the Comptroller of the 2 Currency and then in the private sector involved in five or six 3 turn-around situations with major financial institutions. 4 We'll be covering primarily five areas: the principal 5 reasons a complete historical accounting cannot be rendered, and 6 we'll be looking at some of the specific audits, some of which 7 were ordered by him while he was special trustee. We'll review 8 the advice from a financial institutional advisor to the 9 Department of Interior regarding what they would do in 10 situations where there are missing records. 11 The third area will be we will address the issue of cost 12 and appropriations and the government's sacrificing accuracy and 13 completeness as a result of them. Fourth, dealing with some key 14 definitional issues as to what the accounting statement or 15 accounting should include, and who is to be covered. And 16 finally, the application of certain fundamental trust principles 17 to this matter. 18 PAUL HOMAN, WITNESS FOR THE PLAINTIFFS, SWORN 19 DIRECT EXAMINATION 20 BY MR. DORRIS: 21 Q. Mr. Homan, have you provided an expert report that sets out 22 your qualifications and your background? 23 A. Yes, I have. It's attached as one of the appendices in my 24 expert report, not only this time but in 2003, and it's also 25 briefed in a paragraph in the beginning of my expert report. 1518 1 Q. If we could look on the screen at Plaintiffs' Exhibit 4210. 2 And this is then the cover page of your expert report? 3 A. Yes. 4 Q. And let's look over at the third page, please, sir. And if 5 you would blow up toward the bottom where it has Roman III, 6 qualifications. And is this then where you have set forth your 7 qualifications here? 8 A. Yes. 9 Q. And it briefly summarizes what I had told the Court about 10 your work in both the public service with the Office of the 11 Comptroller of the Currency, and also your work in the private 12 sector, if we can go over to the top of the next page. And the 13 top there -- pause for a moment for the Court to look -- and it 14 also then, the paragraph there talks about your work as a 15 special trustee for American Indians. And were you the first 16 such special trustee pursuant to the 1994 Act? 17 A. Yes. 18 Q. All right. And then at the end, it indicates that there 19 is -- you have a resume which is attached as Exhibit 3 to this 20 report. And let's look at that. If we go to page 390 of 21 Plaintiffs' Exhibit 4210. And if we'd look at the next page, 22 please. And is this a copy of your resume where you have -- 23 that you prepared? 24 A. Yes, I did. 25 Q. And you list first, kind of going back in time, the work 1519 1 that you've done as an expert consultant, and then on the next 2 page, if we can blow that up, your work where you had turnaround 3 planning and experience, and then you go on and list three on 4 this page, including Riggs National Bank here in Washington, 5 D.C., and then three more on the next page where you were 6 involved in turnaround planning and experience. 7 A. That is correct. 8 Q. And then toward the bottom it indicates your work as a bank 9 regulator, experience with the Comptroller of the Currency, and 10 you were there -- this is one particular position that you had, 11 and if we look at the next page, you were with the Comptroller 12 of the Currency from 1966 to 1982, and that's summarized there 13 at the top of the page, correct? 14 A. That is correct. That was my principal experience with the 15 government. I spent the last five years there as the senior 16 career bank official for the office of the Comptroller of the 17 Currency, which supervises the national banks and the largest 18 trust companies in the country. 19 Q. And then it shows your work as a special trustee, and then 20 at the bottom of this page it shows your educational experience; 21 is that correct? 22 A. Yes. 23 MR. DORRIS: Your Honor, I would tender Mr. Homan as 24 an expert in trust management and trust standards and trust and 25 regulatory accounting. 1520 1 MR. QUINN: Your Honor, we have no objection to this 2 witness as an expert in general matters of trust administration. 3 I don't believe the witness has any training or certification 4 here in the accounting, and we would object to his certification 5 or being received as an expert on accounting matters, trust or 6 otherwise. 7 THE COURT: Well, let's hear what his opinions are, 8 and if you want to object to his opinions as he renders them on 9 the grounds that they're not qualified, I'll either consider 10 your objections or let you do voir dire. But let's proceed. 11 MR. QUINN: Thank you, Your Honor. 12 MR. DORRIS: Thank you, Your Honor. 13 BY MR. DORRIS: 14 Q. Let's look at Plaintiffs' Exhibit 4210, your expert report 15 again, on page 4. And if you will -- Your Honor, I would just 16 say, starting here at paragraph Roman numeral V, if you would 17 blow that up -- you will see his separate opinions are separated 18 with the capital letters A, and then it goes on through E, five 19 separate ones, which correlate to the five areas that I 20 indicated to you at the outset I would be covering. And I think 21 we can get started substantially on item A. 22 Mr. Homan, this is then in your report where you've tried 23 to set out your specific opinions; is that correct? 24 A. Yes. 25 Q. The first one, as the Court has an opportunity to look at 1521 1 it, can you state and summarize what you were conveying there? 2 A. Yes. The standards of the Department of Interior in my 3 view, and I believe in the solicitor's view, as expressed to me 4 at least when I was in the Office of the Trustee, is that they 5 must meet the standards of an ordinary trustee. I was the bank 6 supervisor for many years of some of the largest trust companies 7 in the world. I'm aware that in connection with all national 8 bank trust companies, they have been required since the late 9 '70s, along with public trust companies, to keep their books in 10 accordance with GAAP, and to have an independent outside audit 11 conducted annually, using the generally accepted auditing 12 standards. 13 The Department of the Interior has neither. Therefore, I 14 believe that no beginning balance can be certified in any of the 15 accounts. I have reviewed extensively the auditing requirements 16 set forth by the only four audits ever done by the Bureau of 17 Indian Affairs, and later audits, but those were done by Arthur 18 Andersen in 1998. 19 They were all qualified, for the same reasons that the 20 current reports are qualified, and that is that a GAAP audit 21 could not be done because of missing records, lack of prior 22 audits, and accounting and -- trust accounting and trust 23 management systems that were unreliable for decades. 24 MR. QUINN: Your Honor, I object and move to strike. 25 The witness, as we've stated, our objection continuing, witness 1522 1 not qualified to opine on accounting matters. It's not been 2 established he's an expert in the field of accounting. He's 3 testifying to application of standards, generally accepted 4 accounting principles and generally accepted auditing standards. 5 He's not qualified to opine in that regard. 6 THE COURT: Well, he's got a lot of experience, 7 counsel, a lot of experience in a lot of related areas. I'm 8 going to overrule the objection. You can make what you want of 9 it on cross-examination. 10 Proceed, Mr. Dorris. 11 MR. DORRIS: Thank you, Your Honor. 12 BY MR. DORRIS: 13 Q. Mr. Homan, let me try to break some of that down that you 14 just talked about, first of all. First of all, while I think 15 the Court is very familiar with it, you referred to GAAP, which 16 is G-A-A-P, general? 17 A. Accepted accounting principles. And GAAS, which is 18 generally accepted auditing standards. 19 Q. And that's GASS? 20 A. GAAS. 21 Q. And those are standards that you have worked with for many 22 years? 23 A. Yes. And they're also consistent with regulatory 24 accounting principles, which have been used since the '30s by 25 the Office of the Comptroller of the Currency and the other 1523 1 federal banking agencies. They're generally more conservative 2 accounting principles than used in GAAP, although they came 3 together in the late 1990s -- 1970s. 4 Q. So were you a commissioned or certified bank examiner? I 5 can't remember what -- 6 A. I was a certified national bank examiner. 7 Q. Okay. 8 A. Which is the functional equivalent of a CPA, with the 9 additional distinction that bank examiners look not only by law 10 at history, which is what the accountants look at, but also rate 11 banks as to the competence and integrity of management and their 12 future prospects which, according to several financial criteria 13 in common use by the rating agencies in the private sector, and 14 in common use by the bank regulators since 1937. 15 Q. Okay. Let me just, in the first paragraph we have in the 16 screen, under item A, it says, "It was my opinion when I served 17 as Special Trustee for American Indians and it continues to be 18 my opinion today that an accurate and complete historical 19 accounting verified with documentation in conformity with 20 standards set forth in GAAS and GAAP or any other reasonable 21 standard, cannot be rendered." Do you see that? 22 A. That's correct. 23 Q. And you indicated in addition to GAAS and GAAP, you put in 24 there "any other reasonable standard." Why did you say that? 25 A. Well, first of all, there are private sector methods that 1524 1 are consistent with GAAP that are specifically designed to deal 2 with missing records or unreliable trust accounting systems. 3 And they have been used extensively by the public sector in 4 the -- for example, over 2,000 times by the OCC and FDIC since 5 the bank resolutions -- or excuse me, the bank failures from the 6 1970s forward. 7 They've been used in every bank merger case that involves a 8 purchase and assumption transaction and the like. And they're 9 generally described, as I've said just below that, in the 10 methodology proposed by the Bank of America as to how they would 11 handle situations where substantial missing records are missing 12 or where the trust systems, accounting systems, management 13 systems are unreliable. 14 Q. Okay. We'll get to the Bank of America part. You talk 15 about that the opinion you've just given is one that was shared 16 by each of the independent certified public accounting firms 17 that have audited the individual Indian trusts. Give us a brief 18 overview of what audits are available to look at of the BIA or 19 the individual Indian trusts that you're referring to. 20 A. All right. While I was there and subsequently there has 21 been four major categories that I would call attempts at a GAAS 22 or GAAP audit. The first four were conducted by Arthur Andersen 23 from 1988 through 1991. Those were the only audits conducted of 24 the entire Bureau of Indian Affairs' accounting and management 25 systems, since the Office of Trust Fund Management did not exist 1525 1 at that time. They were all qualified, for the reasons I 2 suggested earlier, and essentially for missing records. The 3 fact that the -- 4 Q. I'm just looking for what -- we'll get into the particular 5 ones. 6 A. All right. For the same reasons. 7 Q. You first got Arthur Andersen audits of BIA for four years 8 in the late 1980s. 9 A. That's correct. All right. The next one then was the 10 tribal reconciliation project, also conducted by Arthur 11 Andersen, which attempted to do a GAAP/GAAS audit of the 12 transactions for the tribes in the 1971 to 1992 period. 13 Q. So that's kind of the second item. And you mentioned there 14 were four? 15 A. I was the receiving official on those. The next one was a 16 selected audit by -- an independent outside audit that was 17 required by the Reform Act for the Office of Trust Fund 18 Management, and I received the first one, the second one, the 19 third one, and there has been one every year since. 20 Q. And the first initial ones were done by Griffin as the -- 21 A. That's correct. 22 Q. -- as the auditing company? 23 A. Right. 24 Q. And have you reviewed even the most recent one of those 25 audits, which have continued since you left as Special Trustee, 1526 1 that was done by KPMG? 2 A. Yes. I reviewed each one from 1995, the first one, through 3 the year 2006, although in some cases the management letters 4 that accompanied those, setting forth the specific material 5 weaknesses, I didn't read. But I did read the one in 2006. 6 Q. And are those the four groups that you were referring to 7 earlier? 8 A. No. The fourth one was an audit that I ordered, which was 9 essentially a legal documentation check of some 790,000 OTFM 10 accounts that was conducted in the late '90s while I was there. 11 And it involved bringing in all of the OTFM records, which were 12 the trust accounting records, not the Bureau's records, and we 13 did a legal doc check on each one of those files. 14 Q. And is that what you've referred to in your documentation 15 as the file check? 16 A. That's right. 17 Q. All right. Let's look -- 18 A. It wasn't an audit, per se. 19 Q. It was not an audit in terms of having a CPA come in and 20 look at financial records; is that right? 21 A. Right. 22 Q. All right. Now, if we might pull up Plaintiffs' Exhibit 23 575. And if you would blow that up. This indicates it's the 24 Arthur Andersen tribal and individual Indian monies trust fund 25 audit, financial statements as of September 30, 1989 and 1988. 1527 1 Is this one of the -- is this the first of the Arthur Andersen 2 audits of BIA that you referred to earlier? 3 A. Yes. 4 Q. All right. Let's look then at the next page -- well, let's 5 go one more after that. And let's blow up -- first of all, the 6 date is May 11, 1990. And you can see that it's for the periods 7 that it said on the cover, and let's look down at the bottom two 8 paragraphs of this first page. 9 And in the next to last paragraph on this page, it talks 10 about "The 1988 audit was the first known financial statement 11 audit by independent public accountants of the tribal and 12 individual Indian monies trust funds managed by the Bureau, and 13 the scope of our engagement was not sufficient with respect to 14 the financial statements for preceding periods, from inception 15 through September 30, 1987, to enable us to determine the 16 propriety of trust fund balances resulting from prior activity, 17 and what portions of adjustments identified in our 1988 audit 18 related to prior periods." Do you see that? 19 A. Yes. 20 Q. Now, was this an audit that you reviewed -- while it was 21 dated before you became special trustee, is this something you 22 became aware of and reviewed while you were special trustee? 23 A. Yes. We were interested in what they were -- I was 24 receiving the 1995 audit at the time from Arthur Andersen on the 25 tribal reconciliation. I was interested in what they were able 1528 1 to do in 1988 for essentially the same, covering the same 2 accounts, and -- the tribal accounts back to 1971. So I did 3 review that at that time, and also the notes, which I'm sure 4 you're going to show next. 5 Q. The notes are referred to in that next paragraph, that I 6 will not read aloud, but if you would look at them, it refers to 7 note 3, and then it notes a -- for several reasons, 1 and 2 on 8 that page, with 2 being major inadequacies in the accounting 9 records, and then it goes over to the next page. 10 A. Yes. And what that means to me is that the accounting 11 balances cannot be confirmed. And that goes to the beginning 12 account balances not only of the general ledger but of each 13 subsidiary account, including the IIM subsidiary accounts. 14 Q. And it refers to the Bureau in here a number of times. And 15 who is that? 16 A. The Bureau of Indian Affairs. 17 Q. Okay. So references to the Bureau here is -- 18 A. "BIA," "Bureau of Indian Affairs" are used interchangeably. 19 Q. Let's look at -- you indicated the notes, and we do want to 20 look at the notes. Let's look over -- I think it's page 10 of 21 this exhibit. It's going to be page 9, the one before that, 22 please. And the one before that. So it's page No. 8. 23 And would you highlight toward the bottom there. You 24 recall that, when we were just looking at reference note 3, and 25 I'm now showing you what is note 3 in that financial statement. 1529 1 Do you see it, sir? 2 A. Yes. 3 Q. Okay. Now, would you take a moment and read that first 4 paragraph to yourself? 5 (Witness reviewing document.) 6 A. Yes. 7 Q. And after that -- and we'll look at these pages in just a 8 minute -- there are 16 separate individual items listed there. 9 And I don't want us to look at each of those. But can you 10 summarize what that says to you as someone with your background 11 as a bank examiner in looking at trust departments, what this 12 tells you about a trust? 13 A. These are basic governance and internal controls that were 14 never implemented correctly and appropriately by the Bureau of 15 Indian Affairs. And they go to the weaknesses in staffing, 16 managerial and resource accounting appropriations, if you will, 17 lack of training, and inconsistent trust accounting systems, the 18 inability to do audits, any acceptable audits, GAAS or 19 otherwise. 20 And the burden here is that these accumulate over decades. 21 I believe the Angel report, if I remember, the last testimony 22 documented these types of material weaknesses in accounting and 23 internal controls dating back to the 1930s that were isolated by 24 GAO and the like. 25 I know for a fact, because I looked myself, that the trust 1530 1 accounting systems that were turned over to the Bureau in 1971 2 by the Treasury have been out of balance every year up to 1997, 3 with cumulative general ledger differences that I believe 4 exceeded 105 million, if I remember correctly. 5 Q. And you've indicated this was the first audit by an 6 outside -- 7 A. Well, it describes the first audit. The first audit was 8 done as of 1998. 9 MR. DORRIS: All right. And Your Honor, can we pick 10 up here in the morning? 11 THE COURT: Yeah. That brings us right to 5:00. 12 Thank you, counsel. We will be adjourned until 9:30 tomorrow 13 morning. 14 MR. GINGOLD: Your Honor, I have a question. 15 THE COURT: Yes, sir. 16 MR. GINGOLD: You indicated last week we may go to 17 Friday. If that's the case we'd like to be -- 18 THE COURT: I thought I smiled when I said that. 19 (Laughter) 20 MR. GINGOLD: One point, however. We brought in a lot 21 of witnesses to make sure there are no gaps, so it may mean we 22 may need to bleed over to Friday to finish off the last witness 23 on Thursday, if that's okay. 24 THE COURT: You're going to finish this case on 25 Thursday? 1531 1 MR. GINGOLD: No, Your Honor. But we have a witness 2 that won't be able to come back. 3 THE COURT: Well, we'll talk about this in the 4 morning. I don't think we're going to sit on Friday, but we'll 5 talk about it in the morning. 6 (Proceedings adjourned at 5:00 p.m.) 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 1532 1 INDEX 2 WITNESS: PAGE: 3 Dwight Duncan Direct Examination cont'......1406 4 Cross-Examination.............1452 Redirect Examination..........1512 5 Paul Homan Direct Examination............1517 6 ***** 7 EXHIBITS RECEIVED 8 Plaintiff Exhibit Nos. PPX 4284, 4484 .................. 1450 Plaintiff Exhibit No. PPX 4486 ........................ 1450 9 Plaintiff Exhibit Nos. PPX 4468, 4485 .................. 1452 10 ***** 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25