Multi-tier Caching for High-throughput Indexing Source Code

Background:

Index machines comprise of a cluster or surplus of databases and or file systems. The volume of index data being generated by network connected devices is outpacing data storage technologies’ speed, capacity and abilities. Applications that create index data and generate large volumes of index data. Data deduplication systems is technology made in efforts of carrying out the elimination of redundant data. Successful and efficient data deduplication relies on indexing to maintain performance levels and functionality.

Technology Overview:

Combining key‑value storage and indexing, multi‑tier data structures are able to maintain an index, and maintain performance for databases, database clusters, media tagging and more. An index in a multi‑tier data structure includes providing storage devices to form the structure, caching a list of key‑value pairs, stored on one or more tiers of the multi‑tier data structure. Sub‑lists utilize a caching method with keys, or pointers that reference a block, the data structures currently in use. Interfacing allows keys to manage and control access within individual tiers of these data structures, and point to data stored in the blocks. To manage resources within a multi‑tier storage system and methods for inserting and retrieving key‑value pairs in a machine exist and communicate with multiple storage devices managed in a hierarchy of tiers. Insert and retrieving key values from the machine, transfers sorted lists from memory onto the machine beginning with highest storage tier and then each remaining tier.

Advantages:

Inserting the key‑value pairs in the machine includes transferring a first list of sorted key‑value pairs from a first memory onto the machine to a higher storage tier of the machine, using a merging method. Transferring a second list from the higher storage tiers to the lower storage tiers within the machine creates space in the higher storage tiers. Performing a merging method to redistribute resources within the multi‑tier storage system is aided by the merging method as it is automatically tuned for a workload without disabling the machine.

Applications:

- Computer Science - Computer Architecture - Data & Memory Storage

Intellectual Property Summary:

Patented

Stage of Development:

 

Licensing Status:

Available for License.

Licensing Potential:

Development partner,Commercial partner,Licensing

Additional Information: