Poor Search Methodology Can Waive Privilege

By H. Christopher Boehning and Daniel J. Toal
New York Law Journal, June 20, 2008

Slowly but surely, U.S. Magistrate Judge Paul Grimm is writing a treatise on electronic discovery. In Hopson v. Mayor and City Council of Baltimore, [FOOTNOTE 1] he tackled privilege waiver and warned litigants about the risks of so-called "quick peek" and "clawback" agreements. And in Lorraine v. Markel Amer. Ins., [FOOTNOTE 2] he reminded lawyers to think critically about the admissibility of electronically stored information. In his latest ruling, Victor Stanley Inc. v. Creative Pipe Inc., 2008 WL 2221841 (D. Md. May 29, 2008), Magistrate Judge Grimm revisits the privilege waiver issues at the heart of Hopson and wades into the debate over search methodologies. In the end, his privilege ruling is no surprise and his comments on search methodologies will do little to calm those who are concerned that recent decisions could require litigants to hire experts to defend their chosen search methodology.

In Victor Stanley, the plaintiff sought a ruling that the defendants had waived privilege over 165 electronic documents inadvertently produced during discovery. At the outset of discovery, the parties and their computer forensic experts had met to identify a joint protocol for the search and retrieval of responsive electronic documents. Defense counsel, faced with a substantial amount of electronic data to review, originally requested a "clawback agreement" designed to address the concerns that Grimm discussed in Hopson. [FOOTNOTE 3] When discovery was extended, however, the defendants made the fateful decision to drop this request, instead opting to conduct a full-fledged privilege review.

The defendants endeavored to perform this review on both text-searchable documents (totaling 4.9 gigabytes of data) and non-text-searchable documents (totaling 33.7 gigabytes). With regard to the text-searchable data, the defendants conducted a privilege search using about "seventy different keyword search terms." These search terms were formulated by one of the individual defendants and two attorneys. The attorneys then manually reviewed the documents returned in the privilege search, but completely neglected the remaining documents that the search did not identify as privileged. As to the non-text-searchable documents, the defendants asserted that their vast number compelled a review largely limited to "the page titles." Later, the defendants blamed their inadvertent production of 165 potentially privileged documents on their "compressed schedule and time constraints" in reviewing this non-text-searchable data. The plaintiff, however, pointed out that all of the 165 documents at issue were actually in text-searchable format.

A considerable number of documents that the defendant considered "non-text-searchable" documents were PDF files, "the majority of which were searchable and the remaining could have been made searchable using readily available [Optical Character Recognition] software."

Grimm largely sidesteps the parties' differing characterizations of privilege review, noting that "under either the Plaintiff's or Defendants' version of the events, the Defendants have waived any privilege or protected status for the 165 documents in question." The opinion explains that waiver analysis varies among the federal circuits, ranging from a lenient approach (requiring intentional and knowing relinquishment of the privilege) to a strict approach (typically finding waiver because a party cannot restore confidentiality once lost). While the U.S. Court of Appeals for the Fourth Circuit, like the Second Circuit, has yet to definitively adopt an approach to analyzing waiver, Grimm applies the intermediate approach used in the district courts of both the Fourth and Second circuits to evaluate the issue. [FOOTNOTE 4]

This intermediate approach requires balancing several factors: "(1) the reasonableness of the precautions taken to prevent inadvertent disclosure; (2) the number of inadvertent disclosures; (3) the extent of the disclosures; (4) any delay in measures taken to rectify the disclosures; and (5) overriding interests in justice."

The linchpin of the analysis was the first factor -- the reasonableness of the defendants' privilege review. Observing that the defendants "bear the burden of proving their conduct was reasonable," Magistrate Judge Grimm held that they failed to meet this burden. Of particular concern was the defendants' failure to explain their search methodology. The defendants, he stressed, "failed to provide the court with information regarding: the keywords used; the rationale for their selection; the qualifications of [search designers] to design an effective and reliable search and information retrieval method; whether the search was a simple keyword search, or a more sophisticated one, such as one employing Boolean proximity operators; or whether they analyzed the results of the search to assess its reliability, appropriateness for the task, and the quality of its implementation."

Balancing Act

After quickly resolving that the other waiver factors favor the plaintiff as well, Grimm offers some guidance to litigants faced with the unnerving high-wire act of performing a thorough electronic privilege search within the time constraints set by the court while also trying to keep expenses under control. The choice of a particular information search and retrieval product and the formulation of the search itself require careful advance planning.[FOOTNOTE 5] For assistance, lawyers, not necessarily adept at formulating searches that "involve[] technical, if not scientific knowledge," should first turn to the best practices identified by the Sedona Conference. [FOOTNOTE 6]

Formulating, Implementing a Search: Victor Stanley instructs that a poorly conceived or cursory privilege review is insufficient to prevent waiver of privilege in cases of inadvertent document disclosure. The opinion highlights five best practice points from the Sedona Conference that lawyers should adopt in order to formulate a search methodology that can later be justified to a court. The Sedona Conference encourages parties to:

  • formulate a search with reference to the specific legal context;
  • perform due diligence in choosing a particular search product;
  • recognize that using an information retrieval tool does not guarantee that all responsive documents will be identified;
  • make a good faith attempt to cooperate on choosing and implementing information retrieval; and
  • expect that their choice of methodology will need to be explained. [FOOTNOTE 7]

The defendants' inability to protect their clients' privileged information was a direct result of their failure to heed these best practice points. Defendants did not appreciate the inherent complexity of the electronic discovery process, understand the nature or format of the electronic documents they reviewed, or verify the results of their work. They also inexplicably withdrew their request for a clawback agreement.

Explaining the Search Rationale: In addition to confirming what should have been obvious to any practitioner -- that a poorly designed keyword search would not suffice to overcome a claim of waiver -- Victor Stanley adds to the growing number of decisions calling upon lawyers to demonstrate that the search methodology chosen was appropriate. Here, when instructed to supply support, the defense attorneys, who (along with their client) selected the keywords used for searching, offered nothing.

The defendants did not offer a defense of the search terms chosen or provide background to establish their qualifications "for designing a search and information retrieval strategy that could be expected to produce an effective and reliable privilege review." [FOOTNOTE 8]

Those omissions were compounded by the fact that the defendants had failed to sample the results of their privilege search, leaving them with no indication of effectiveness, when "[c]ommon sense dictates that sampling and other quality assurance techniques must be employed to meet requirements of completeness." [FOOTNOTE 9] This led the court to conclude that the keyword searching relied upon by the defendants was insufficient, because "simple keyword searches end up being both over- and underinclusive in light of the inherent malleability and ambiguity of spoken and written English (as well as all other languages)." [FOOTNOTE 10]

That Grimm would require defense counsel to explain its search methodology and the rationale underlying it is unexceptional. But in so doing, he raises some interesting questions about how a party should explain its methodology. The judge, although cautious in his dicta, raises the specter of requiring expert testimony to support search methodology. In a lengthy footnote, he delves into two opinions by U.S. Magistrate Judge John M. Facciola of the U.S. District Court for the District of Columbia, U.S. v. O'Keefe [FOOTNOTE 11] and Equity Analytics, LLC v. Lundin. [FOOTNOTE 12]

These opinions proceed from the premise that "determining whether a particular search methodology ... will or will not be effective certainly requires knowledge beyond the ken of a lay person (and a lay lawyer)." [FOOTNOTE 13] And as such, "this topic ... requires that any conclusion be based on evidence that, for example, meets the criteria of Rule 702 of the Federal Rules of Evidence." [FOOTNOTE 14]

O'Keefe and Equity Analytics unnerved some commentators, who worry that "engraft[ing] Rule 702 ... into discovery would multiply the costs of discovery." [FOOTNOTE 15] Magistrate Judge Grimm, however, reads his colleague's opinions in a rather narrow manner. He observes: "Viewed in its proper context, all that O'Keefe and Equity Analytics required was that the parties be prepared to back up their positions ... with reliable information from someone with the qualifications to provide helpful opinions."

So, although the requisite "reliable information" may include a "qualified expert," it is not so limited. In summarizing his position, Grimm again specifically notes that in choosing a methodology, parties should be "aware of literature describing the strengths and weaknesses of various methodologies, such as The Sedona Conference Best Practices." Then, if the search methodology is ultimately challenged, the party "should expect to support their position with affidavits or other equivalent information from persons with the requisite qualifications and experience, based on sufficient facts or data and using reliable principles or methodology."

The takeaway point seems to be that when formulating a search, a party must have some cogent rationale. If the methodology cannot be supported with reference to industry standards, a treatise, or other previously tested rationales, an expert may be required to overcome the burden of showing that the search was reasonable.

Need for Transparency: Above all, the opinion in Victor Stanley highlights the need for transparency and collaboration in the e-discovery process. [FOOTNOTE 16] As Grimm notes, the parties should confer and agree on a search and retrieval method and thus "minimize cost because if the method is approved, there will be no dispute resolving its sufficiency."

Lawyers should identify the information that exists, determine the sources and format of the information, consider the burdens and costs of production, and come to an agreement on a reasonable search protocol to be approved by the court. Throughout this process, lawyers must recognize that the search for electronic documents is an inexact science; inevitably, no search methodology will produce every responsive document or identify every potentially privileged one.

Conclusion

Victor Stanley highlights the need for lawyers to think carefully about their approach to e-discovery. The search for electronic information is not a task that can be accomplished quickly or haphazardly. The protection of client confidences requires that lawyers carefully plan and implement their searches, mindful of the potential need to explain their methodology to the court; and that they work with opposing counsel to identify a search methodology that will be effective and cost-efficient.

H. Christopher Boehning and Daniel J. Toal are litigation partners at Paul, Weiss, Rifkind, Wharton & Garrison. Daniel S. Kirschbaum, an associate at the firm, and Jeffrey J. Recher, a summer associate, assisted in the preparation of this article.

Footnotes

FN1 232 F.R.D. 228 (D. Md. 2005). For a discussion of Hopson, see John F. Baughman and H. Christopher Boehning, "'Hopson' Is a Good Reminder That There Are No Short Cuts," NYLJ (Feb. 28, 2006).

FN2 2007 WL 1300739 (D.Md. May 4, 2007).

FN3 See 232 F.R.D. at 235.

FN4 For cases in the Second Circuit that adopt a balancing test consisting of several factors, see Hydraflow Inc. v. Enidine Inc., 145 F.R.D. 626 (W.D.N.Y. 1993); Local 851 of the Int'l Brotherhood of Teamsters v. Kuehne & Nagel Air Freight Inc., 36 F. Supp.2d 127 (E.D.N.Y. 1999); Large v. Our Lady of Mercy Medical Center, No. 94 Civ. 5986 (JGK)THK, 1998 WL 65995 (S.D.N.Y. Feb. 17, 1998).

FN5 For a discussion by the authors of the recent literature and findings on various search methodologies, see H. Christopher Boehning and Daniel J. Toal, "Assessing Alternative Search Methodologies," NYLJ (April 22, 2008).

FN6 See Victor Stanley, 2008 WL 2221841 at *6 (citing The Sedona Conference Best Practices Commentary on the Use of Search & Information Retrieval Methods in E-Discovery, 8 Sedona Conf. J. 189 (2007)).

FN7 See id. (citing 8 Sedona Conf. J. at 194-95, 201-02).

FN8 Id. at *3.

FN9 Id. at * 5 (quoting In re Seroquel Prods. Liab. Litig., 244 F.R.D. 650, 662 (M.D. Fla. 2007)).

FN10 Id. (quoting 8 Sedona Conf. J. at 194-95).

FN11 537 F. Supp. 2d 14 (D.D.C. 2008).

FN12 248 F.R.D. 331 (D.D.C. 2008).

FN13 Equity Analytics, 248 F.R.D. at 333.

FN14 O'Keefe, 537 F. Supp. 2d at 24.

FN15 Ronald J. Hedges, Rule 702 and Discovery of Electronically Stored Information, 8 Digital Discovery and E-Evidence (BNA) No. 5, at p. 4 (April 11, 2008).

FN16 See also John F. Baughman and H. Christopher Boehning, "Benefits of Transparency: Discussing What Will Be Produced With Your Adversary," NYLJ (April 25, 2006).

Source: i-fact@nalysis, Inc.
Website: http://i-factanalysis.com/

USA Offices:
Boston | Houston | Los Angeles | Miami | New York | San Francisco | Seattle |
Washington, D.C.
International Offices:
Hong Kong | London | Mexico| | Mumbai | Riyadh | Shanghai | São Paulo
Terms of Use | Privacy Policy | Site Map | Contact Us | Headquarters:
225 Franklin Street, Ste 2600, Boston, MA 02110 | 617.217.2850
Copyright © 2016 i-fact@nalysis Inc. All Rights Reserved.

Site implementation by Setting Sun Design.