Technology - Disk-Resident Streaming Dictionary

Disk-Resident Streaming Dictionary

Computer program product for storing data in a disk storage system utilizing efficient and optimal insertion speed.

Background:

Modern networks utilize and generate high‑bandwidth streams of data, often creating the issue of data being produced at a faster rate, as compared to the ability of data being inserted into a database. Many industries face the data stream issue, as it is easier to collect data at a high rate, but they are not necessarily able to have indexing and searching of this data at an efficient rate. Almost all database or file systems employ a data dictionary mapping keys to values.

Technology Overview:

A dictionary is an approach to tackling the data stream issue, as it essentially provides a mapping from keys to values. Keys are totally ordered, using a comparison function. A dictionary can be thought of as containing key‑value pairs, upon providing a key, the system can find the key’s associated value in the dictionary. Within this dictionary, the key system is the ability to perform a range of scans on the dictionary where all key‑value pairs are easily accessible. Implementations for this technology include but are not limited to a buffered B‑tree and a cascading array, alongside a transactional support for the dictionaries. The particular embodiment of a method of storing data in a disk storage system includes defining a dictionary data structure stored on the disk storage system.

Please note, header image is purely illustrative. Source: geralt, pixabay, CC0.

Advantages:

This method for organizing data in a disk storage system provides efficient access of data, perhaps influencing factors of performance, multiple versions of data can easily be maintained. Key value pairs, placed in nodes structured as a tree, utilize a pointer as the pivot key for faster access of data. The system of the invention can be distributed between many software processes on several data communication devices, or all processes could run on a small set of dedicated computers.