Towards a cleaner Banking environment

Author: Arvind Kamath
Business Intelligence Lead with EDS, HP
It would not be an exaggeration to say that the sector in which change has been most visible in past few years is banking. The new paradigm of doing business changed the rules of game not only for what is commonly known as new economy but also for age old established business. The driver for this change has in most cases been the regulatory requirements and the advancements in technology making it more and more capable of helping banks in meeting these requirements.

Data Quality is one of the most important aspects of a Bank’s Data Management efforts and one of the most challenging too. The high volume of data coupled with complex nature in a Banking ecosystem is thanks to modern technology driven entities that generate massive amount of data through their multiple channels of businesses. With the advent of ERP, CRM and SCM systems to support line operations of organizations, data generation has reached mind boggling levels in recent years.

Banking data is characterized by huge volume of business transactions, maintenance of history records and the need to have multiple, disparate applications for various banking functions. The stringent statutory requirements in Banking, present vastly different needs for data management than any other industry vertical.

Sources of major challenges faced by the banking industry from the point of view of data management can be categorized under following heads:
* Data quality
* Legacy applications
* Volume of data
* Statutory Requirements
* Security

The author, with his extensive experience and background in banking domain, Data warehousing and Business Intelligence, proposes to draw on his knowledge and suggest solutions to the data management problems faced by the banking industry with respect to data quality.

The paper would help in understanding the problems faced by the banking industry in particular and will help Data warehousing consultants suggest specific and focused solutions.

Data Quality Management in Banking
Quality of data as a challenge to better IT management is not unique to banks. However when looked in conjunction with other factors e.g. Legacy applications, real time usage, volume of data and not the least geographic spread of operations the severity of challenge becomes very critical.

What is Data Quality?
Data Quality can be defined as fitness or suitability of data to meet business requirements. The various attributes that collectively characterize quality of data are:
* Accuracy
* Integrity
* Consistency
* Completeness
* Validity
* Timeliness
* Accessibility

Data need not be perfect, but simply needs to meet the requirements of applications that use it.

Why is it required?
Unreliable data is the prime reason why Data Quality is required coupled with data entry related errors, duplicate records and missing or incorrect data values.

What does it involve?
Data Quality involves:
a. Fixing of defective data elements, records and existing incorrect values (e.g. fix a misspelling).
b. Modifying data to correct value to conform to corporate standard (e.g. Substitute “Mr.” for “Mister”), replacing a missing value, match and merge of duplicate records that exist in same file.
c. Filtering of data, which involves deleting duplicate, missing or nonsensical data

Which is the best place to clean data?
The best place to clean data is in the source application system.

Other places to clean data are a) staging area, b) during the Extract Transform and Load (ETL) process and c) in the Data Warehouse itself.

Criticality of Data Quality in Banking
The various reasons why data quality is so critical in Banking:
* Increased use of business intelligence systems across companies for various Decision Support Systems
* Since Banks have to meet stringent regulatory requirements defined by the Central Bank, the reports have to be accurate and this is possible only if data is clean
* Ensuring clean Customer data and maintaining a single view of the Customer across databases is a big challenge facing the Banks. Banks incur very high cost of erroneous mailings especially in the Unites States of America and the resultant effect of missing targeted customers
* Rapid growth of eCommerce, which has resulted in multiple entry points
* Banks are expected to collect historical data to comply with Basel II requirement. The historical data needs to be clean to meet the highest data standards prescribed by Basel II accord.
* Focus on complying with regulation such as Sarbanes Oxley, has also forced Banks to come up against the problem of finding and interpreting the required data
* Banks are increasingly deploying Risk management applications for tracking, monitoring and analysis for operational and other risks. Effective analysis would depend on how clean the data is.

Other benefits of Data Quality in Banking
The Management’s confidence in the various MIS reports generated from the Banking decision support system would increase multi folds, if a proper and efficient data quality program is in place.

The time spent by the Accounts department in reconciling data generated from multiple systems would be reduced drastically. The main reason why data reconciliation is required is only because of duplicate records and unclean data.

With clean customer data, the Bank can effectively perform cross selling activity and increase profitability.

Indirectly, as discussed elsewhere, Banks would reduce cost on communication. A single customer may receive same mail more than once if proper customer house holding is not done and also the probability of missing the attention of a Customer if his address details are not clean in the system.

Sources of unclean data in Banking
The sources of unclean data in banking applications are due to:
* Integration with multiple databases across wide geographies across disparate systems
* Mergers and acquisitions happening in the Banking industry have resulted in integrating multiple systems and databases and chances of duplication of records
* Use of paper documents to input data into the system, which may result in wrong data entering the system both from the writer’s as well as data entry operator’s perspective
* Slowly changing dimension e.g. change in the name profile of a Customer post marriage, change of address etc
* Using home grown data cleansing tools, which may meet only defined data quality problems but not those that may creep into the system due to dynamic banking environment
* Due to widespread use of Internet banking technology, the Banking systems are prone to direct usage by Customers for entering their personal details and other information. Purposeful entry of incorrect data by Customers affects data quality.
* The operators at call centers may enter incomplete data to save on time
* Data entering the core banking application from various sources affect data quality due to inconsistent data format or data errors.
* If Banks rely on third party for data capturing, the errors in their data files would result in bad data entering the Bank’s system

Data Quality and Decision Support Systems
With the advent of business intelligence tools specific to Banking industry, Banks are investing heavily into this new technology for decision support. The pre-requisite for deploying such tools is data warehousing. Data warehousing is implemented since Banks deploy multiple applications around its ecosystem and hence the need for single version of truth.

Unclean data in any of the source applications or systems would affect the warehouse and thereby the analytical reports. The millions spent on data warehousing and business intelligence tools would go down the drain if it is not able to give reliable output.

Decisions based on such output could mar the prospects of the bank especially in the areas of Risk management and Asset Liability management, which often decides the course of action Bank may take

Investment on clean data campaign would certainly benefit the Bank and also justify the implementation of data warehousing and decision support systems. The money spent on such a campaign is miniscule compared to the benefit it would derive in the long run.

Hence today most of the Banks are forced to invest in data quality tools along with the data warehousing initiative to derive good mileage

If Banks have to survive in the present competitive world where retaining existing Customers and acquiring new ones is paramount to its sustenance and growth, it is but imperative that they have to depend heavily on advanced analytics, business intelligence and CRM initiatives.

To achieve rapid return on these decision support systems, data quality should be planned as a proactive initiative rather than a cause and effect event.

Banks may spend more on ensuring operational efficiency than on data warehousing and CRM initiatives if a defined data quality program is not in its agenda.

Data Quality program in Banking
The Bank’s IT team has to draw up a data quality plan within its overall data management program. It is never too late for the Bank if such an initiative does not exist, as delay could only result in serious operational problems for the Bank.

Since Banks collect data from multitude of sources, which are not standard or inconsistent, certainly requires a strong data quality process.

The Bank must answer the following basic questions before undertaking a data quality program:
1. Is data quality to be ensured at source or after the data is extracted into the data warehouse from multiple and disparate sources?
2. How often should data be checked for quality?
3. Which are the applications that require stricter adherence to data quality than others? This is only to prioritize the applications as bringing all of them together could hamper the progress of the initiative
4. Which are the target systems or applications which require absolutely clean data?
5. Which are the systems or applications that contain high volume of data compared to others?

Data quality initiative should never be undertaken as an after effect of an unprofitable and bad marketing campaign or after losing customers.

Banks also depend heavily on third party sources for data like address data, customer information data for their credit worthiness, market quotes for treasury applications etc. It is imperative that all data may not be clean and would require cleansing before it enters the Bank’s database.

Steps for a successful Data Quality program in Banking
Bank should first conduct an audit of its database with help of external IT consultants to know the extent of data quality.

Data quality program should be synchronous with any planned data management program like data warehousing, data mining etc.

A team of business analysts from the IT team together with the Subject Matter Experts or SMEs should be formed for the Discovery phase. The SMEs should be drawn out from various application/domain areas, which would be covered in the data quality initiative.

Before a Data quality project is really initiated it is suggested that Bank follow these steps in partnership with its technology consultants.

The author of the article is Arvind Kamath, a Business Intelligence Practice Lead with EDS, HP.
Previous  article
Next article
Write your comment now

Email    Password: 
Don't have SiliconIndia account? Sign up    Forgot your password? Reset
Reader's comments(3)
1: From: Mrs. Mary David

This mail may be a surprise to you because you did not give me the permission to do so and neither do you know me but before I tell you about myself I want you to please forgive me for sending this mail without your permission. I am writing this letter in confidence believing that if it is the will of God for you to help me and my family, God almighty will bless and reward you abundantly. I need an honest and trust worthy person like you to entrust this huge transfer project unto.

My name is Mrs. Mary David, The Branch Manager of a Financial Institution. I am a Ghanaian married with 3 kids. I am writing to solicit your assistance in the transfer of US$7,500,000.00 Dollars. This fund is the excess of what my branch in which I am the manager made as profit last year (i.e. 2010 financial year). I have already submitted an annual report for that year to my head office in Accra-Ghana as I have watched with keen interest as they will never know of this excess. I have since, placed this amount of US$7,500,000.00 Dollars on an Escrow Coded account without a beneficiary (Anonymous) to avoid trace.

As an officer of the bank, I cannot be directly connected to this money thus I am impelled to request for your assistance to receive this money into your bank account on my behalf. I agree that 40% of this money will be for you as a foreign partner, in respect to the provision of a foreign account, and 60% would be for me. I do need to stress that there are practically no risk involved in this. It's going to be a bank-to-bank transfer. All I need from you is to stand as the original depositor of this fund so that the fund can be transferred to your account.

If you accept this offer, I will appreciate your timely response to me. This is why and only reason why I contacted you, I am willing to go into partnership investment with you owing to your wealth of experience, So please if you are interested to assist on this venture kindly contact me back for a brief discussion on how to proceed.

All correspondence must be via my private E-mail ( for obvious security reasons.

Best regards,
Mrs. Mary David.
Posted by: mary lovely david - Monday 26th, September 2011
2: Hi my dear,
My name is Mounace, i would like to establish a true relationship with you in one love. please send email to me at ( i will reply to you with my picture and tell you more about myself. thanks and remain blessed for me,
Your new friend Mounace
Posted by: mounace love love - Thursday 09th, June 2011
3: Arvind do you think CRM can used as a system for a paper-less environment in banks?
Posted by: Priscilla Thosago - Wednesday 30th, June 2010
More articles
by Kaushal Mehta - Founder & CEO, Motif Inc..
The retail industry is witnessing an increased migration of customers from traditional brick and mortar retail to E-commerce (online retail)...more>>
by Samir Shah - CEO, Zephyr .
You probably do because you are on the phone with them! For all of you working in some technical management capacity here in Silicon Valley,...more>>
by Raj Karamchedu - Chief Operating Officer, Legend Silicon .
These days are a mixed bag for me. Of late I have been considering "doing something bigger and better," in my life, perhaps seriously though...more>>
by Madhavi Vuppalapati - CEO of Prithvi Information Solutions .
IT Services Rise of Tier II companies The Indian IT outsourcing industry is going through very exciting phase in its business life...more>>
by Bhaskar Bakthavatsalu- Country Manager, India and SAARC of Check Point Software Technologies.
Data loss occurs every day through corporate email. In fact, given the sheer number of emails an organization sends every day, data loss inc...more>>