Will the Real ECA Please Stand Up?
There is a tremendous amount of confusion regarding what early case assessment (ECA) is and isn't. There are some traditional attorneys and litigation support personnel maintaining that their collection of worksheets and interviews is the real ECA. At the same time there are a number of e-discovery software vendors claiming that their latest product (or re-packaged old product) is the real ECA. Working in the e-discovery field nearly every day, I've come to believe that both sides of the argument have their own merits, pitfalls, and limitations, and that the answer is somewhere in-between tradition and technology.
"Traditional" ECA
Being an old school counterintelligence guy, I know the value of going out and talking to people versus relying on technology to provide you all the answers in a tidy package. Checklists and interviews with individual custodians and IT personnel can get you to the proper data stores that you need, but there are some potential pitfalls to this approach:
Often times the individuals interviewing custodians or IT personnel do not have an adequate understanding of the myriad of enterprise, virtualized, open source, data storage, clustered, custom, middleware, out-sourced, and dozens of other buzzword solutions that can exist in the average computing environment. This is the time that you really want to have your experienced data collection professionals with you to make sure that each area of the IT environment is mapped out and all of its data stores are understood.
Checklists are a great way to make sure that you don't forget to ask critical questions. Unfortunately, a checklist doesn't know how to ask a follow-up question, nor have I ever seen an ECA checklist that will work for every situation. Strict adherence to a checklist will very likely result in an incomplete collection. I'm in favor of using a continuously-evolving checklist that starts at a high-level (example: "Do you outsource any IT services?") and then lets the interviewer burrow down into more specific questions as you get a feel for the IT environment. A good checklist is part of a set of living documents that change as technology and cases change.
People's memories are fallible. Duties and personnel within a large IT environment change over time as much as the technology changes. Computers, applications and data stores get left behind and forgotten on a regular basis in a rapidly changing IT infrastructure. This is especially true in situations where companies have merged or been purchased. This situation is where technology and follow-up interviews become important as the picture of the data environment becomes clearer.
All that being said, I still find that a good, long discussion with key IT personnel, along with a number of follow-up discussions, is one of the best ways to understand how and where data is stored and used within an organization. There are things that simply cannot be learned or discovered by running a search algorithm or relying on a computer to provide the right answers.
A collection will also go much more smoothly and will be more cost effective if you have a knowledgeable data collection specialist helping during the ECA interview process. An attorney best understands the language and situations faced by other attorneys; the same principle applies to IT personnel. Only years of experience in a multitude of IT environments can prepare one to collect the most complete information from other IT professionals.
"Software / Hardware Solution" ECA
The other side of this discussion involves delving into a number of e-discovery vendors and their ECA software that purports to solve all your problems and save you truckloads of money. I always ask people that have been disappointed by an e-discovery software vendor that promised them software that does the impossible: "What else have you ever bought in your whole life that has fixed all of your problems, and saved you money?" Just like most software out there, e-discovery software has to be approached with a healthy dose of skepticism. Many of the e-discovery software packages available today that claim to perform ECA require that you collect the data first (a prime example of cart before the horse) and then index it so that it can be searched, or they require that some specialized server be attached to the targeted network so that it can "crawl" the network. Some of the problems with these technologies are:
The flaw inherent in the "index and search" ECA tools is that you have to know where the data of interest is, and what format it's in, so that it can be collected before you index and search it. If you knew where the right data was, what format it was in, and how to properly collect it, you would have already done some great ECA. According to the EDRM model, indexing and searching should fall neatly into the areas of Processing, Review and Analysis, which comes after Identification, Preservation and Collection.
Collecting and dumping all of the data into an index and search ECA tool is the opposite of saving money. Most of the "index and search" ECA tools charge on what has become known as "The Clearwell Model." This pricing model is based on the amount of data being indexed. In this "pay to index" model, it is in the vendor's best financial interest to push as much data into the system as possible simply to generate the highest bill possible. That certainly doesn't sound like a recipe for saving money during ECA.
ECA software that is delivered on server hardware to "crawl" the prospective network for data suffers from similar licensing fee issues, typically charging by the amount of data indexed or by the number of computers reviewed. To make matters worse, these ECA "crawler" solutions are limited in the types of data stores they can access, suffer from access control and file permission problems, and have additional issues in dealing with complex networks with multiple physical or logical domains.
I'm not proposing that every ECA software vendor is selling a bad product or trying to run up a huge bill. What I am saying is to make sure that you have someone who is experienced in e-discovery collection examine the solution and confer with you on what the tool is actually going to be able to collect prior to committing. A collection professional will be able to assist with deciphering the pricing model for the ECA software solution, how the solution works, and how it deals with complex data stores like databases, mail servers, ERP solutions and access controls.
Something that ECA software does very well is fill a niche in an area that's been buzzing recently: "first pass review." This process is essentially the same thing we've been calling "indexing and searching" for years, but I do love a good buzzword. In situations where an attorney or document reviewer wishes to review data prior to any kind of culling (such as de-duplicating, applying data ranges, de-NISTing, etc.), many of the ECA software solutions would serve quite well. A good use of a first pass review tool might be to collect and examine the data from a few custodians to better understand how data is stored and used prior to full-scale collection. However, be sure to mind the costs while doing something like that, as they may quickly match or even exceed what it would cost to just perform the full collection.
Final Thoughts
While I've been talking about the ECA situation with clients and groups of attorneys for a while now, I often feel like I am shouting into the wind against all the ECA software vendor advertising that I receive nearly every day. I was gratified to read "Don't Box ECA. Early case assessment precedes e-discovery." by George Socha and Tom Gelbmann (founders of the EDRM Reference Model and the Socha-Gelbmann Electronic Discovery Survey). In their article on Law.com, Socha and Gelbman write "In our opinion, the term early case assessment is misapplied in the e-discovery context. While ECA can - and often should - address e-discovery issues, early case assessment sweeps much more broadly."
Keep in mind that the news on ECA software isn't all bad. We've started to see a growing number of vendors producing point solutions that address collection of complex data from systems like Microsoft SharePoint and Microsoft SQL Server. There are also a small group of vendors offering email archiving solutions that can be used to search and extract minimal sets of email rather than whole mailboxes or mail stores. Lastly, there is a growing set of desktop and laptop-level collection tools that can be used to perform limited data collections based on pre-determined data types, date ranges and keyword sets.
As always, before purchasing any of these new ECA solutions, prospective buyers should find out all the details of the pricing models and test the product to make sure it will actually collect data as promised and in the specific way you need it collected. We've tested several solutions that don't live up to their marketing material. When armed with the right people and tools, however, a properly performed ECA can positively affect the outcome (and cost) of your matter.
If you find something for ECA that works well and is priced right, please feel free to share it with us. Or, if you have a new product that you think is "the next big ECA thing," as an "agnostic" e-discovery and computer forensic firm we are always happy to provide unbiased testing.
