Enron Corpus

The Enron Corpus is a database of over 600,000 emails generated by 158 employees[1] of the Enron Corporation in the years leading up to the company's collapse in December 2001. The corpus was generated from Enron email servers by the Federal Energy Regulatory Commission (FERC) during its subsequent investigation.[2] A copy of the email database was subsequently purchased for $10,000 by Andrew McCallum, a computer scientist at the University of Massachusetts Amherst.[3] He released this copy to researchers, providing a trove of data that has been used for studies on social networking and computer-mediated communication.

Creation

In the legal investigation into Enron's collapse, the discovery process required collecting and preserving vast amounts of data, for which the FERC hired Aspen Systems (now part of Lockheed Martin). The emails were collected at Enron Corporation headquarters in Houston during two weeks in May 2002 by Joe Bartling,[4] a litigation support and data analysis contractor for Aspen. In addition to the Enron employee emails, all of Enron's enterprise database systems,[5] hosted in Oracle databases on Sun Microsystems servers, were captured and preserved, including its online energy trading platform, EnronOnline.

Once collected, the Enron emails were processed and hosted in proprietary electronic discovery platforms (first Concordance, then iCONECT) for review by investigators from the FERC, Commodity Futures Trading Commission, and Department of Justice. At the conclusion of the investigation, and upon the issuance of the FERC staff report,[6] the emails and information collected were deemed to be in the public domain, to be used for historical research and academic purposes. The email archive was made publicly available and searchable via the web using iCONECT 24/7, but the sheer volume of email of over 160GB made it impractical to use. Copies of the collected emails and databases were made available on hard drives.

Jitesh Shetty and Jafar Adibi from the University of Southern California processed the data in 2004 and released a MySQL version.[7] In 2010, EDRM.net published a revised and expanded version 2 of the corpus,[8] containing over 1.7 million messages, which has been made available on Amazon S3 for easy access to the researchers.

Exploitation

The corpus is valued as one of the few publicly available mass collections of real emails easily available for study; such collections are typically bound by numerous privacy and legal restrictions which render them prohibitively difficult to access, such as non-disclosure agreements and data sanitization.[3] Shetty and Adibi, based on their MySQL version, published some link analysis of which user accounts emailed which.[9] Linguistic comparison with more recent email corpora shows changes in the email register of English. It is also used as test or training data for research in natural language processing and machine learning.[10]

References
  1. Klimt, Bryan; Yiming Yang (2004). "The Enron Corpus: A New Dataset for Email Classification Research": 217–226. CiteSeerX 10.1.1.61.1645. Cite journal requires |journal= (help)
  2. "The Enron Email Corpus Archived 2011-03-08 at the Wayback Machine" Retrieved March 5, 2011.
  3. Markoff, John. "Armies of Expensive Lawyers, Replaced by Cheaper Software". New York Times March 5, 2011. p A1.
  4. Bartling, Joe (September 3, 2015). "The Enron Data Set - Where Did It Come From?". Bartling Forensic and Advisory. Retrieved September 3, 2015.
  5. "FERC: Industries - Enron's Energy Trading Business Process and Databases". www.ferc.gov. Retrieved 2015-09-02.
  6. FERC Staff Report - Price Manipulation in Western Markets - Findings at a Glance (3-26-2003)
  7. "Enron processed database"
  8. Socha, George. "EDRM Enron Email Data Set v2 Now Available". EDRM.net. Archived from the original on 2011-09-04. Retrieved 2012-09-03.
  9. Shetty, Jitesh; Adibi, Jafar (2005). "Discovering important nodes through graph entropy the case of Enron email database". Proceedings of the 3rd international workshop on Link discovery - LinkKDD '05. pp. 74–81. doi:10.1145/1134271.1134282. ISBN 978-1595932150.
  10. Friginal, Eric; Hardy, Jack (2013). Corpus-Based Sociolinguistics: A Guide for Students. Routledge. p. 167. ISBN 978-1-136-29277-4. Retrieved 29 May 2020.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.