Raima Database Manager

Raima Database Manager (or RDM) is an ACID-compliant embedded database management system designed for use in embedded systems applications. RDM has been designed to utilize multi-core computers, networking (local or wide area), and on-disk or in-memory storage management. RDM provides support for multiple application programming interfaces (APIs): low-level C API, C++, and SQL(native, ODBC, JDBC, ADO.NET, and REST). RDM is highly portable and is available on Windows, Linux, Unix and several real-time or embedded operating systems. A source-code license is also available.

RDM
Developer(s)Raima
Stable release
14.2
Operating systemCross-platform
TypeDBMS
LicenseCommercial license
Websiteraima.com/raima-database-manager/

RDM has support for both non-SQL (record and cursor level database access) and SQL database design and manipulation capabilities. The non-SQL features are important for the most resource-restricted embedded system environments, where high performance in a very small footprint is the priority. SQL is important in providing a widely known standard database access method in a small enough footprint for most embedded systems environments.

History

Raima Inc. originally released RDM in 1984 and it was called db_VISTA. It was one of the first microcomputer network model database management systems designed exclusively for use with C language applications. A companion product called db_QUERY was introduced in 1986, which was the first SQL-like query and report writing utility for a network model database.

A db_VISTA derivative DBMS designed to provide a high-performance, transaction-processing client-server SQL DBMS called Raima Database Server (RDS) was released in 1993. This was the first DBMS that provided an ODBC API as its native SQL interface. It was also the first SQL system that incorporated use of the network model in its DDL features. Soon thereafter, RDS was renamed Velocis and in 2001, RDM Server. Version 8.4 of RDM Server was released in 2012.

Uninterrupted development of RDM (also known as RDM Embedded) has continued, with the most recent feature additions including database mirroring use in support of highly available (HA) systems, database replication, multi-version concurrency with read-only transactions, multiple transactional file server access, encryption, and an SQL designed specifically for use in embedded systems applications. Version 12.0 of RDM was released in 2013.

Also in 2013, RDM introduced the first on-platform SQL DBMS available for use with National Instruments' LabView graphical programming language; it was named the National Instruments' LabView Embedded Tools Network Product of the Year.[1]

Version 14.0 of RDM was released in Q3/16. RDM v. 14.0 contains an all-new data storage engine optimized specifically for working with in-memory resident data sets. The new in-memory database (IMDB) allows for significant performance gains and a reduction in processing requirements when compared to older in-memory or on-disk implementations.

Version 14.1 of RDM was released in Q1/18. The new release focuses on ease of use, portability, and speed. With Raima's new file format, you can develop once and deploy anywhere. Performance is increased by over 50-100% depending on the use case when compared to previous RDM releases. Raima has extended and improved SQL support, snapshots, and geospatial functionality.

Version 14.2 of RDM was released in 2020. The new release has continuous focus on ease of use, portability, and speed. Multi-user Focused Storage Format: the updated database file format increase database throughput through a focus on locking contention prevention. Extended and improved geospatial functionality and a newly supported REST-ful interface have been added to the database server functionality.

Product features

Both the source code lines and features in Raima Database Manager and RDM Server are consolidated into one source code. RDM includes these major features: updated in-memory support, snapshots, R-Tree support, compression, encryption, SQL, SQL PL, and platform independence—develop once, deploy anywhere. RMD includes portability options such as direct copy and paste that permit development and deployment on different target platforms, regardless of architecture or byte order. The release includes a streamlined interface that is cursor-based, extended SQL support and stored procedures that support SQL PL; it also supports ODBC (C, C++), ADO.NET (C#), RESTful and JDBC (Java). Supported development environments include Microsoft Visual Studio, Apple XCode, Eclipse and Wind River Workbench. A redesigned and optimized database file format architecture maintains ACID compliance and data safeguards, with separate formats for in-memory, on-disk or hybrid storage. File formats hide hardware platform specifics (e.g., byte ordering). Download packages include examples of RDM speed and performance benchmarks.

Transactional File Server (TFS)

A software component within the RDM system that maintains safe multi-user transactional updates to a set of files and responds to page requests. The TFServer utility links to the TFS to allow it to run as a separate utility, allowing users to run RDM in a distributed computing environment. The TFS may also be linked directly into an application to avoid the RPC overhead of calling a separate server.

Modes of Operation

  • Single-Process, Multi-Thread
  • Multi-Process, separate Transactional File Server
  • Multi-Process, shared in-process Transactional File Server

Dynamic DDL

  • Support for on-the-fly alterations of the database and tables themselves

Encryption

  • AES 128, 192, 256 bit

Additional SQL Data Types

  • Date
  • Time
  • Datetime
  • Binary
  • Unicode
  • Bit

Data Providers and Drivers: Interoperability

  • ADO.Net 4.0 Data Provider
  • JDBC 4.2 Type 4 Driver
  • ODBC 3.51 Driver
  • RESTful API

Different “tree” support

Snapshots

  • Snapshot isolation allows concurrent reads to the database when write transactions are occurring. RDM takes a frozen image of the current state of the system and that information can be read from without stopping writes. At any point in time, the user can issue a snapshot of specific tables through calling our rdm_dbStartSnapshot() API. Once done, the RDM system will create a static view of the tables specified where any changes to those tables will not be reflected in the snapshot. The user is then free to issue writes to that table outside of the snapshot and any reads within the snapshot view will not be waiting for those writes to complete or preventing those writes from finishing. Once the snapshot is no longer needed, a simple end transaction can be called to easily and quickly get rid of it. This feature provides the end user with the largest number of writes and reads possible simultaneously.

Circular Tables support

  • A record type, or table, can be defined as “circular.” With circular tables, when the table becomes full, RDM will still allow new record instances to be created. The new record instances will overwrite existing ones, starting with the oldest. RDM does not allow explicit deletion of record instances in a circular table.
  • The definition of a circular table includes a size limit. This provides a useful way of allocating a fixed amount of storage space for storing the most recent instances of a particular record type. For example, this may be useful in storing event data that is being generated rapidly, where only the most recent data is relevant. Circular tables remove the risk that incoming data may fail to be stored due to lack of space, while avoiding the need for the application to delete obsolete data.

Database limitations

  • Maximum Databases Open Simultaneously: No Limit
  • Maximum Records Per Database: No Limit
  • Maximum Size of Database File: Limited only by file system
  • Maximum Tables Per Database: No Limit
  • Maximum Records Per Table: No Limit
  • Maximum Record Size: 32 kb (excluding BLOB or VARCHAR)
  • Maximum Fields Per Table: No Limit
  • Maximum Keys Per Database: No Limit
  • RAM Requirements: User-configurable, minimum 50 kb
  • Code Footprint: Starting at ~270 kb, depending on OS and database features

Data Types Supported

  • BLOBs
  • Character
  • Widechar
  • Varchar
  • DBADDR (ROWID)
  • Floating Point – 32 bit and 64 bit
  • Integer – 8 bit, 16 bit, 32 bit and 64 bit
  • C Struct (Core only)
  • Data/Time/Timestamp
  • BCD (SQL Decimal) – Binary-code-decimal is a standard database representation for financial applications.
  • GUID

Database Design Language (DDL)

Non-SQL (core) DDL Features:

  • C struct-like record type (table) declarations.
  • Network model set declarations for defining 1-many inter-record relationships.
  • Support for direct, B-tree, and hashed record access.
  • In-memory database or file declarations. A database can be designed to be either on-disk or in-memory, or a hybrid where some parts reside in-memory while others are stored on disk.
  • Circular record types (tables). Circular tables store a user-specified maximum number of records (rows). When that maximum has been reached, newly inserted records are stored in the location occupied by the oldest one. Circular tables are important for storing status data on resource-restricted devices.
  • Supported datatypes: 8, 16, 32, and 64 bit signed or unsigned integers, float, double, decimal (BCD), fixed or variable-length character or wide character, binary or character large objects (blobs), date, time, timestamp, guid/uuid, and db_addr (database address—aka, rowid).
  • Support for struct and array data fields.
  • Optional user control over database file organization and page sizes.

SQL DDL Features:

  • Declared referential integrity support automatically implemented using RDM's network model sets.
  • Support for direct, B-tree, and hashed row access.
  • In-memory database or table declarations.
  • Circular tables.
  • Virtual tables declarations that provide SQL access to external data sources (e.g., real-time sensor data).
  • Supported data types: boolean, tinyint, smallint, integer, bigint, decimal, real, float/double, binary/varbinary, long varbinary, char/varchar, wchar/wvarchar, long varchar, long wvarchar, date, time, timestamp, guid/uuid, rowid (foreign and primary keys).
  • Domain declarations.

Transactional File Server

The RDM Transactional File Server (TFS) specializes in the serving and managing of database files on a given medium. The TFS is a set of functions called by the RDM runtime to manage the sharing of database files among one or more runtime library instances. In a normal multi-user configuration, the TFS functions are wrapped into a server process called TFServer. Standard TCP/IP can be used to make the connection, whether the runtime library and TFServer are on the same computer or different computers. However, when on the same computer, a faster, shared-memory protocol is available by default.

The figure shows that one RDM client runtime may have connections to multiple TFServers, and one TFServer may be used by multiple client runtimes. To the applications using the RDM runtime and the TFServers, the locations of the other processes are invisible, so all processes may be on one computer, or all may be on different computers. This provides opportunities for true distributed processing.

Raima Database Manager Transactional File Server Operation

A TFServer should be considered a “database controller” in much the same way as a disk controller manages a storage device. A TFS is initialized with a root directory in which are stored all files managed by the TFS. If one computer has multiple disk controllers, it is recommended that one TFServer be assigned to each controller. This facilitates parallelism on one computer, especially when multiple CPU cores are also present.

A complete application system may have multiple TFServers running on one computer, and multiple computers networked together. Each TFServer will be able to run in parallel with the others, allowing the performance to scale accordingly.

The TFS functions are used by the RDM runtime, so the programmer has no visibility of the calls made to them. These functions are made available to the runtime library in three forms. For descriptive reasons, we call them TFSr, TFSt and TFSs:

  • TFSt: The actual, full-featured TFS functions, called directly by the runtime library. Supports multiple threads in a single application.
  • TFSr: The RPC (Remote Procedure Call) library. When called by the runtime library, these functions connect to one or more TFServer processes and call the TFS functions within them. A client/server configuration.
  • TFSs: “Standalone” TFS functions called directly by the runtime library, but intended only for single-process use (if multiple threads are used, each must be accessing a different database only). To be used for high-throughput batch operations while the database(s) are otherwise offline. Unsafe (but fast) updates are allowed.

Database unions

RDM's database union feature provides a unified view of multiple identically structured databases. Since RDM allows highly distributed data storage and processing, this feature provides a mechanism for unifying the distributed data, giving it the appearance of a single, large database.

As a simple illustration, consider a widely distributed database for an organization that has its headquarters in Seattle, and branch offices in Boston, London and Mumbai. Each office owns and maintains employee records locally, but the headquarters also performs reporting on the entire organization. The database at each location has a structure identical to the others, and although it is a fully contained database at each location, it is also considered a partition of the larger global database. In this case, the partitioning is based on geographical location. The mechanism for querying a distributed database is simple for the programmer. When the database is opened, all partitions are referenced together, with OR symbols (“|”) between the individual partition names.

Partitioning and unified queries are also used for scaling the performance. Consider a database where each operation begins with a lookup of a record’s primary key. If the“database” is composed of four partitions, each stored on the same multi-core computer, but on different disks controlled by different disk controllers, then the only requirement is a scheme that divides the primary key among the four partitions. If that scheme is a modulo of the primary key, then the application quickly determines which partition to store a record into or read the record from. Since there are multiple CPU cores to run the multiple processes (both the applications and the TFSs), and the four partitions are accessible in parallel (the four controllers permit this), the processing capacity is four times larger than with a single-core, single-disk, and single-partition configuration.

Database encryption

RDM allows all database content to be encrypted before it is transported across a network and written to the database files. RDM's encryption supports the Rijndael/AES encryption algorithm with 128-, 192- or 256-bit keys based on an application-specified encryption key.

Database Mirroring and HA Support

Database mirroring in RDM reproduces an exact, byte-for-byte copy of a master database onto the mirrored (or slave) database. Database mirroring is an important database feature for applications that require high availability (HA) where should a TFServer fail for some reason, then the application's HA monitor can automatically switch over to the mirrored TFServer.

RDM provides synchronous mirroring where each transaction that is committed on the master TFServer is also securely committed to the mirror TFServer. RDM also provides a set HA support API functions that can be called from the application's HA monitor to monitor the operational status of the TFServers.

Mirroring can also be used to support maintaining multiple copies of a database in which updates are only made to the master but readers are directed to one of the mirrored slaves in order to distribute many possible database readers across multiple computers. In this situation, it is not necessary for the master to wait for each slave to confirm a successful commit of each transaction and the mirroring process can run asynchronously.

RDM database mirroring requires that the master and all mirrored databases be maintained on the same computer/operating system platforms.

Database replication

This is due to be released in Q2/18. Replication is similar to mirroring but it not really intended for HA support but for transferring all or, more likely, portions of one database (master) to another database (slave). Replication is designed to work where the databases are not necessarily being maintained on the same platform. The slave databases can be other RDM-managed databases or they can be a 3rd-party DBMS.

RDM's replication includes support for multiple master to single slave selective replication of circular table data—important for embedded computers and devices at the edge of the data grid where status and condition monitoring occurs. The status data stored in each master's circular table is replicated to a central control system maintaining a permanent history of all device statuses, which can then be made available for a variety of time series and other analyses.

RDM also provides a database change notification API library that allows a slave to access the master replication logs without the data be stored and managed in a database. This allows, for example, a master to store device control information in a database that is replicated to the device through the notification API in order to efficiently control device operation.

SQL/PL

The RDM SQL Programming Language (SQL PL) is based on the ansi/ISO SQL Persistent Stored Modules (PSM) specification (ISO/IEC 90756-4:2011 +2012). It provides a high-level language in which stored procedures and functions can be written, compiled and called within the RDM SQL system.

  • SQL PL is a computationally complete programming language for use in RDM SQL stored routines (procedures or functions). The language is block structured with the ability to declare variables that conform to the usual scoping rules with an assignment statement so that value can be assigned to them. Control flow constructs provided include if-elseif-else and case statements along with several loop control constructs (including while, repeat-until, and for loop statement).
  • Seamless access to SQL is provided via the ability to execute most SQL statements that can include references to locally declared variables. Also provided is the ability to declare cursors allowing rows from select statements to be fetched into locally declared variables, allowing the result column values to be checked and manipulated within the stored routine.
  • Exception handling is also provided, allowing handlers to be coded for specific or classes of errors or statuses returned from the execution of an SQL statement. In addition, it is also possible to define a user condition and exception handler and for the program to signal its own, special-purpose exceptions.

RDM SQL has been designed specifically for use in embedded systems applications. Some of the more important features of RDM SQL include:

  • Small footprint—no SQL views or security is provided as these are usually unnecessary in embedded systems apps and their absence helps to keep the SQL footprint small.[2]
  • Standard SQL transaction and referential integrity support.
  • The SQL system catalog and stored procedures can be stored in a file or as statically declared data structures in C modules.
  • Cost-based query optimization with a rich set of built-in scalar and aggregate functions. A variety of table access methods are available for consideration by the optimizer: direct row access (through rowid primarykeys), optimal primary/foreign key join access through network model sets, B-tree and hash indexes.
  • Ability to extend SQL capabilities through C-based user-defined scalar and aggregate functions.
  • Ability to extend SQL capabilities through C-based user-defined virtual table interfaces that provide SQL access to external data sources such as real-time sensor data.
  • Database table import/export to/from comma-delimited or XML files.
  • Ability to have read-only access from SQL to a non-SQL (i.e.,core level) database. This means, for example, that a remote RDM SQL application can access a non-SQL RDM database running on a very resource-restricted device.

Application programming interfaces

RDM provides application programming interfaces that allow application development in a variety of programming languages:

  • C based Cursor API – Facilitates the traversal of database records for retrieval, insertion, update and removal of database records. With Record, Key and Set cursors, it fits seamlessly with RDM’s database concepts. It resembles modern programming concepts of iteration over a collection.
  • Comprehensive SQL API – Accessed internally through a simplified ODBC-like API that uses a Raima design. It also supports stored procedures and most other standard SQL.
  • SQL Programming Language (PL) API – Allows programming logic to be done through pure SQL. Developers can leverage their knowledge of SQL and still add programming conditionals and logic.
  • Standards Based ODBC API – Following the ODBC standards, the ODBC API was developed so developers have a familiar way to use the RDM database engine.
  • JDBC – Standard JAVA interface to the RDM database engine, with two modes of operation: the first through TCP/IP and the second a direct link through JNI.
  • ADO.NET – Standard C# interface. Supported connection method is through TCP/IP.
  • RESTful – The REST-ful API is a modern API designed for application developers who want to be able to view and modify database contents through HTTP GET, POST, PUT and DELETE methods with a return format of JSON. Additionally, an administrative set of APIs is exposed to allow for a quick overview of the whole RDM subsystem's status. Items such as memory usage, CPU usage, database size, and database configuration are all available through the HTTP interface. This API is perfect for the developer who is interested in web development or wants to create a quick interface to an RDM database that is accessible on any platform through a web browser.
  • Object Oriented C++ Cursor API – The C++ API was designed to be easy to use while providing developers with full access and control to both RDM’s network and relational functionality.
  • Legacy Navigational C API –RDM’s low-level C is still supported, with minor changes required for the developer.

Supported platforms

RDM has been ported to a wide variety of computers and operating systems. Packages are available for the following platforms:

RDM packages

RDM consists of two packages: RDM Core and RDM Enterprise.

  • RDM Core includes just the core cursor API interface; it is the underlying and most-optimized API designed for use with the C programming language.
  • RDM Enterprise contains both the core cursor API and the SQL interface, in addition to all of the remaining APIs. This package allows for the use of the C# ADO.NET interface, the JAVA JDBC interface, RESTful API and the ODBC interface. It also has full support for third-party connectivity and administrative tools, in addition to supporting the full legacy API from previous versions of the RDM product line.

Customers and applications

RDM based applications are used today in all major industries including Aerospace & Defense, Automotive, Business Automation, Financial, Government, Industrial Automation, Medical, and Telecommunication. A sampling of RDM users includes the following:

  • Mitsubishi Electric—iQ Platform C Controller PLC
  • Schneider Electric—"ezXOS" in OASyS DNA product
  • Hydro-Québec—CEDA system to manage set up and configuration of power plant alternators
  • General Dynamics—"TIEF" – Tactical Information Exchange Capability database agent
  • Boeing—"AWACS"– Airborne Warning and Control System's radar electronics system
  • Raytheon—Low-level tactical flight profile management in Pave Hawk
  • Lockheed Martin—Flight simulators
  • Benu Networks—Broadband Service Delivery Platform
  • Johnson & Johnson—VITROS patient systems
  • Beckman Coulter—UniCel DxC 800 Synchron clinical system
  • Siemens—RapidPoint 400 medical fluid test equipment
  • IBM—ClearCase source code control system
  • Magellan Navigation—MAPSEND GPS used in PC-based and embedded products
  • NSE—Reliable stock trade data storage
  • NCDEX—Real-time database services for trading application

References
  1. Laura Arnold (2013-05-08). "Top Alliance Partner Achievements Recognized During NIWeek 2013 - Discussion Forums - National Instruments". Decibel.ni.com. Retrieved 2017-06-25.
  2. Randy Merilatt (2010-06-09). "Is using SQL in an embedded computer application like trying to squeeze an ELEPHANT into a MINI? - Embedded Computing Design". Embedded-computing.com. Retrieved 2017-06-25.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.