siliconindia | | August 20168Shahid ShahIN MY OPINIONLAMBDA ARCHITECTURE--IDEAL FOR BIG DATA USE CASESShahid Shah, Co-Founder & CEO, Netspective Communications>business rules is often called early-binding. Early-bound data is quite useful for use cases where the data sources, structures and associated business rules don't change very often.Since DMWs have pre-defined data structures, their database schemas are set before data is written into the database. This is often referred to as schema on write because the type, format, and rules for data is known in advance, when the data is stored in the database. Once a data warehouse is created and data is written into it, its ability to change formats, types, or most other attributes is limited at best. Much of the time spent in managing DMWs is in the extract, transform, and load (ETL) process and then once that process is in place then the analysts using the data in the warehouses are stuck with the dimensions created by the ETL process. Any kind of reporting or investigation of data that changes the format, style, units of measure, etc. would require going through the ETL process again. This is why DMWs are considered enterprise-grade, usually have a high cost to setup, and a high cost to maintain.Since the technologies, architectures, approaches, and designs for DMWs are well understood, they are good for use by talented and experienced business analysts looking to perform well defined, but seldom changing, business processes that produce retrospective analytics and reports. However, traditional DMWs cause long term maintenance and user challenges when data scientists need to do ad hoc or exploratory data discovery which is a necessity for true value-based payment models. DMWs are great when Most of the pressing problems in healthcare cannot be solved without agile multi-stakeholder and multi-institution data integration. Early-bound data marts and warehouses are not flexible enough so we need to deploy late-binding "Lambda Architecture" style data structures.The Affordable Care Act (ACA), Medicare Access & CHIP Reauthorization Act of 2015 (MACRA), Merit-Based Incentive Payment System (MIPS), Alternative Payment Models (APMs), Precision Medicine Initiative (PMI), and Patient-Centered Outcomes Research through PCORI are all taking us towards a more value-driven payment system for the U.S. healthcare system. Physicians and hospitals have been, for decades, paid fees for services they perform on patients and the higher their volume the more money they made (regardless of outcomes). Given the unsustainable growth rates in national healthcare spending, all health insurers and the federal government are working to figure out how to pay providers and health systems for the value they deliver to patients and the public health system.ACA, MACRA, MIPS, APMs, PMI, PCORI, and the many other initiatives the healthcare industry has embarked upon all have an insatiable appetite for data. Unfortunately, existing data architectures built on analytical data marts and data warehouses are starting to prove insufficient when asked to handle complex next generation value-based business models which require more collaborative and flexible data processing.Why Data Marts and Warehouses are InsufficientToday's data infrastructure was built for a world in which health providers get paid for almost any services applied to almost any patient without regard to outcomes. Data marts and warehouses (DMW) are great for pre-structured and pre-processed data sources that are coming from a small, relatively fixed or slowly changing, number of transactional systems. The assumption that we know our sources of data and their formats along with their
< Page 7 | Page 9 >