Hybrid transactional/analytical processing

Hybrid transaction/analytical processing (HTAP), a term created by Gartner Inc. – an information technology research and advisory company. As defined by Gartner:

Hybrid transaction/analytical processing (HTAP) is an emerging application architecture that "breaks the wall" between transaction processing and analytics. It enables more informed and "in business real time" decision making.[1][2]

Background

In the 1960s, computer use in the business sector began with payroll transactions and later included tasks in areas such as accounting and billing. At that time, users entered data, and the system processed it at a later time. Further development of instantaneous data processing, or online transaction processing (OLTP), led to widespread OLTP use in government and business-sector information systems.[3]

Online analytical processing (OLAP) covers the analytical processing involved in creating, synthesizing, and managing data. With greater data demands among businesses, OLAP also has evolved. To meet the needs of applications, both technologies are dependent on their own systems and distinct architectures.[4][3] As a result of the complexity in the information architecture and infrastructure of both OLTP and OLAP systems, data analysis is delayed.[4]

HTAP advantages and challenges

There are various interpretations of HTAP other than Gartner's original definition; an "emerging architecture". These interpretations suggest different advantages, one being a database functionality. Recent advances in research, hardware, OLTP and OLAP capabilities, in-memory and cloud native database technologies,[5] scalable transactional management and products enable transactional processing and analytics, or HTAP, to operate on the same database.[4][6][3]

However, Gartner's most recent reports suggest broader advantages than a single unified database can offer. Traditional application architectures separated transactional and analytical systems. Digital business, and the need to respond to business moments, means that using "after the fact" analysis is no longer adequate. Business moments are transient opportunities that must be exploited in real time. If an organization is unable to recognize and/or respond quickly to a business moment by taking fast and well-informed decisions, then some other organization will, resulting in a missed opportunity (or a new business threat). HTAP allows advanced analytics to be run in real time on "in flight" transaction data, providing an architecture that empowers users to respond more effectively to business moments.[7]

The main technical challenges for an HTAP database are how to be efficient both for operational (many small transactions with a high fraction of updates) and analytical workloads (large and complex queries traversing large number of rows) on the same database system and how to prevent the interference of the analytical queries over the operational workload. This kind of operational workload is also commonly referred to as Operational Analytical Processing.

HTAP solves the issue of analytic latency in several ways, including eliminating the need for multiple copies of the same data and the requirement for data to be offloaded from operational databases to data warehouses via ETL processes.[4][6]

Most applications of HTAP are enabled by in-memory technologies that can process a high volume of transactions and offer features such as forecasting and simulations. New HTAP technologies use scalable transactional processing, and do not need to rely on keeping the whole database in-memory. HTAP has the potential to change the way organizations do business by offering immediate business decision-making capabilities based on live and sophisticated analytics of large volumes of data. Government and business leaders can be informed of real-time issues, outcomes, and trends that necessitate action, such as in the areas of public safety, risk management, and fraud detection.[4][8]

Some challenges for HTAP include limited industry experience and skills, as well as undefined best practices.[4]

HTAP functionality is offered by database companies, such as Microsoft Azure Synapse Link[9] for Cosmos DB, DbAlibaba DRDS, LeanXcale,[10] TiDB,[11][12] Hubble, ArangoDB, Aerospike, Apache Ignite/GridGain In-Memory Data Fabric, IBM IBM_Db2 IDAA,[13] InterSystems,[14][15] Kdb+, Microsoft SQL Server, Neo4j, Oracle 12c In-Memory,[16] SAP HANA,[17][18] MemSQL, MongoDB, VoltDB, NuoDB, OrientDB, DataStax, eXtremeDB, Splice Machine,[19] EsgynDB, Cloud Spanner, HarperDB, Amazon Aurora (Parallel Query), BlobCity, Couchbase,[20] YugabyteDB[21] and Postgres.

References
  1. "Market Guide for HTAP-Enabling In-Memory Computing Technologies". www.gartner.com. Retrieved 15 April 2017.
  2. "Hybrid Transaction/Analytical Processing Will Foster Opportunities for Dramatic Business Innovation". www.gartner.com. Retrieved 15 April 2017.
  3. Bog, Anja. Benchmarking Transaction and Analytical Processing Systems: The Creation of a Mixed Workload Benchmark and Its Application Springer-Verlage Berlin Heidelberg. 2014
  4. Pezzini, Massimo; Feinberg, Donald; Rayner, Nigel; Edjlali, Roxane. "Hybrid Transaction/Analytical Processing Will Foster Opportunities for Dramatic Business Innovation." Gartner. 28 January 2014
  5. "Azure Analytics: Clarity in an instant". azure.microsoft.com. Retrieved 20 June 2020.
  6. Wolpe, Toby. "SQL and NoSQL? Fine, but how does the hybrid database fit in?" ZDNet. 12 May 2014
  7. "How to Enable Digital Business Innovation via Hybrid Transaction/Analytical Processing". www.gartner.com. Retrieved 15 April 2017.
  8. Baer, Tony. "Fast Data hits the Big Data fast lane." ZDNet. 16 April 2012
  9. "A closer look at Azure Synapse Link". www.zdnet.com. Retrieved 15 April 2017.
  10. Research., Bloor. "Hybrid real-time data processing". bloorresearch.com. Retrieved 30 October 2019.
  11. "The Hybrid Database Capturing Perishable Insights at Yiguo". Datanami. 22 February 2018. Retrieved 2 March 2018.
  12. Xu, Kevin. "How TiDB combines OLTP and OLAP in a distributed database". InfoWorld. Retrieved 14 November 2018.
  13. "Real Time Analytics with IDAA and Big Data on System z"
  14. Inc., Gartner. "Operational Database Management Systems (ODBMS) Software Reviews". Gartner. Retrieved 14 February 2018.
  15. Gartner (12 February 2018). "Critical Capabilities for Operational Database Management Systems". Gartner.
  16. "Leading-edge Database technology now available in all environments". oracle.com. Archived from the original on 28 August 2018.
  17. Review, CIO. "Internet of Everything and Hybrid Transactional Analytical Processing". CIOReview. Retrieved 26 March 2016.
  18. "Gartner Reprint". www.gartner.com. Retrieved 26 March 2016.
  19. "The Splice Machine Data Platform". Splice Machine.
  20. https://www.datanami.com/2018/09/20/couchbase-to-deliver-parallel-json-analytics-without-the-etl/
  21. https://docs.yugabyte.com/latest/faq/general/
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.