New Advances in Enterprise Software for Next Generation Content Archiving

Author: Barry Murphy
Director - Product Marketing, Mimosa Systems
The commercially available email archive solutions perform the basic task of collecting email and storing them in an indexed database for fast search and retrieval. The data collection methods used by email archival solutions determine the breadth of information retained and the impact on Exchange Servers.

Challenges of Traditional Email Archiving Applications

At a basic level, commercially available email archiving applications provide an indexed database to catalog records, search and retrieve them, and apply security. Archiving applications reduce the burden of managing old email and allow users to access the archive for search and retrieval instead of querying the messaging server. Typical comparisons of email archiving products delve into the details of storage, search, and security and pay little attention to what method is used to collect data from the messaging servers. The data collection methods vary by the richness of message information they collect and the extent to which they impact Exchange performance.

Simple Mail Transfer Protocol (SMTP) Data Collection is a favorite method of email archival service providers. SMTP Data Collection intercepts email at the gateway server, while the message is in SMTP format. This method has the advantage of collecting email at a single point. And because SMTP is compatible with all message servers, it supports all the popular email types (e.g., Microsoft Exchange and IBM Lotus Notes). All data collection processing is performed on the gateway and does not burden the messaging server.

The drawback of SMTP Data Collection is that it is either limited to inbound or outbound messages only or it forces all internal messages also to be relayed through the gateway, thereby increasing the load on the messaging server. SMTP also contains only basic information for date, time, from, to, cc, subject, body, and attachment. Beyond this basic content information, there exists rich information that relates to the context and lifecycle of the message, such as folder location, flags, rich text, settings, replies, forwards, edits, opens, deletes, and folder changes.

Microsoft Message API (MAPI) is the Microsoft supported method for reading messages in Microsoft Exchange Server. Traditional email archive solutions use MAPI to scan messages in the Message Store, copy the messages to the archive, and index them for fast search and retrieval.

The major drawback of MAPI is the performance burden it places on the Exchange Server. To lessen this burden, email archiving solutions restrict the amount of information collected for each message. Each message contains its header, body, and attachment (if any), and over 400 individual properties that contain the message context and lifecycle information of each message. Because of the time MAPI takes to read the message content, the message properties are not collected. This reduces the overall burden of MAPI on the Exchange Server but limits the amount of information available for regulatory and legal discovery analysis.
The second method that is used to limit the burden on the Exchange Server is to schedule the archiving application to run on the weekend, when email use is low. The drawback of this scheduled approach is that messages sent, received, and deleted during the week are not available for archiving. Only messages that exceed a certain age (e.g., 60 days) and still remain in mailboxes are copied to the archive server. Organizations that must retain 'all email' for compliance and run MAPI Data Collection on a weekly basis use Exchange Journaling to capture email traffic during the week.

Microsoft Exchange Journaling archives all messages sent and received on any given Message Store. The drawback of Journaling is the CPU and storage burden it puts on the Exchange. Journaling captures one additional copy for each message sent or received and forces internal messages to go through the expensive route that outbound messages take even if the recipients are on the same server. To manage Journaling for compliance, email archiving solutions use MAPI to read the Journal mailboxes and truncate their contents. Without MAPI, the Journaling mailboxes would quickly fill and consume valuable Exchange storage space.

The second drawback of Journaling is its lack of rich message information. The Journal mailbox is a stripped down version that bears no similarity to the assigned mailbox except for the content of the messages. Journaling only views the message when it is sent or received; otherwise it loses all contact with the message and its history. Journaling is not able to capture the rich lifecycle information contained in the user's mailbox (open, replies, forwards, etc.) — information that are extremely valuable to internal investigations and eDiscovery.

Moving Forward
The next generation of archiving applications needs to address the drawbacks of the traditional approaches described above, provides cost effective, efficient archiving that retains the rich context and lifecycle information of each message; complies with regulatory requirements; enables legal staff and investigators to easily search and review message content and enforce litigation holds; and minimize the storage burden.
Previous  article
Next article
Write your comment now

Email    Password: 
Don't have SiliconIndia account? Sign up    Forgot your password? Reset
Reader's comments(2)
1: From: Mrs. Mary David

This mail may be a surprise to you because you did not give me the permission to do so and neither do you know me but before I tell you about myself I want you to please forgive me for sending this mail without your permission. I am writing this letter in confidence believing that if it is the will of God for you to help me and my family, God almighty will bless and reward you abundantly. I need an honest and trust worthy person like you to entrust this huge transfer project unto.

My name is Mrs. Mary David, The Branch Manager of a Financial Institution. I am a Ghanaian married with 3 kids. I am writing to solicit your assistance in the transfer of US$7,500,000.00 Dollars. This fund is the excess of what my branch in which I am the manager made as profit last year (i.e. 2010 financial year). I have already submitted an annual report for that year to my head office in Accra-Ghana as I have watched with keen interest as they will never know of this excess. I have since, placed this amount of US$7,500,000.00 Dollars on an Escrow Coded account without a beneficiary (Anonymous) to avoid trace.

As an officer of the bank, I cannot be directly connected to this money thus I am impelled to request for your assistance to receive this money into your bank account on my behalf. I agree that 40% of this money will be for you as a foreign partner, in respect to the provision of a foreign account, and 60% would be for me. I do need to stress that there are practically no risk involved in this. It's going to be a bank-to-bank transfer. All I need from you is to stand as the original depositor of this fund so that the fund can be transferred to your account.

If you accept this offer, I will appreciate your timely response to me. This is why and only reason why I contacted you, I am willing to go into partnership investment with you owing to your wealth of experience, So please if you are interested to assist on this venture kindly contact me back for a brief discussion on how to proceed.

All correspondence must be via my private E-mail ( for obvious security reasons.

Best regards,
Mrs. Mary David.
Posted by: mary lovely david - Monday 26th, September 2011
My Name is Tata I was impressed when i saw your profile at and will like you to email me back to my inbox so that i can send you my picture for you to know who i am.i belive we can establishe a long lasting relation ship with you.In addition,i will like you to reply me through my private e mail box (
This is because i dont know the possibilities of
remainning in forum for a long time.
Thanks,waiting to hear from you soonest.
Posted by: tata tatababy os - Friday 30th, October 2009
More articles
by Kaushal Mehta - Founder & CEO, Motif Inc..
The retail industry is witnessing an increased migration of customers from traditional brick and mortar retail to E-commerce (online retail)...more>>
by Samir Shah - CEO, Zephyr .
You probably do because you are on the phone with them! For all of you working in some technical management capacity here in Silicon Valley,...more>>
by Raj Karamchedu - Chief Operating Officer, Legend Silicon .
These days are a mixed bag for me. Of late I have been considering "doing something bigger and better," in my life, perhaps seriously though...more>>
by Madhavi Vuppalapati - CEO of Prithvi Information Solutions .
IT Services Rise of Tier II companies The Indian IT outsourcing industry is going through very exciting phase in its business life...more>>
by Bhaskar Bakthavatsalu- Country Manager, India and SAARC of Check Point Software Technologies.
Data loss occurs every day through corporate email. In fact, given the sheer number of emails an organization sends every day, data loss inc...more>>