IndexedDB: Keys: efficiently retrieving data from a database

In order to retrieve data efficiently from an IndexedDB database, each record is organized by its key. The only condition for the key is that it is a valid key. A valid key can be one of the following types:

• Array (Is only valid when every item inside the array is a valid key and the array doesn’t contains it self. Also all non-numeric properties are ignored and will not affect whether the array is a valid key)
• DOMString
• Date (The Primitivevalue (internal property) of the date object can’t be NaN)
• Float (The key can’t be NaN)

When comparing keys, the order in the list above applies. Array is greater than all DOMStrings, DOMString is greater than …

Comparing arrays is done by comparing the values inside the array (as long that both arrays got values on the position). If the values on the same position differ, then the outcome of that comparison will determine which array is greater. If the values on each position are the same, the length of the array will determine the which one is greater or equal if the arrays have the same length.

Complex Keys

The IndexedDB API also support complex keys. This are keys who refer to properties nested inside an object. This way you are able to filter on data that is available are properties of a nested object. In the example below we have an person object. Besides a name and firstname property it contains an address property. This address property is also an object containing properties like city, street, … If we want to filter data on its city we can do this by defining a key like this “address.city”.

   1: var person = {

   2:     name: "name",

   3:     firstname: "firstname",

   4:     address: {

   5:         street: "street",

   6:         city: "city"    

   7:     }

   8: }

Since Linq2IndexedDB 1.0.11 this is also supported for all filters, even for filters that don’t take advantage of the IndexedDB API. For this reason it is advised to use the linq2IndexedDB.prototype.utilities.getPropertyValue method to determine the value by it’s property name. Two arguments need to be provided:

• first argument is the object containing the data
  
• second argument is the propertyName string. For example “address.city”

The return value of the method is the value at this location if it exists, otherwise undefined will be returned.

Note that this nesting only works for objects. If the property contains an array of objects, you won’t be able to use this way to filter your data.

Object store keys

Inside an object store, keys even have a more special reason of existence. In here keys are used to uniquely identify an object. This means you can never add an object twice with the same key, but this enables you to update and delete objects when you know it’s key. For an object store, there are 2 ways you can provide a key:

• Inline key: the key is present inside the object (A KeyPath must be defined on the object store)
  
• External key: the key is provided separately (in most cases the value of the key/value will be a primary type like a string or float)

In both cases you can choose for the option to let the key be generated by the IndexedDB API. You can do this by setting the object store autoIncrement attribute on true when creating the object store. In this case the key will have a numeric value.

Index keys

The keys in for an index are always defined by a KeyPath. These keys can be complex keys as defined in the section “Complex keys” above. Every object in the object store keeping the index that has a value for the key path a record will be added. The key will be the value of the property defined by the key path. For example we have a keyPath “firstname”. In the example below, the key in the index record would be “Kristof” and the value the person object.

   1: var person = {

   2:     name: "Degrave",

   3:     firstname: "Kristof"

   4: }

In the index you can also have 2 special cases. The first one is the unique attribute. This will make sure, that a value for this keyPath can only be defined once in a value of the object store. This constraint will be checked when you add data to an object store. If there is an object present with the same value for that property, an ConstraintError will be thrown.

A second attribute is the multiEntry. This allows to have multiple values for only one key. In stead of adding a new record for every object with the same value, the key is only added once and all the objects that have same value are added to a collection.

Conclusion

Now you should have insight how the keys work inside the IndexedDB API. With the complex keys you will be able to filter on data with nested object, this way you don’t need to provide an object store for every object you want to store. Also the Linq2IndexedDB framework also supports these complex keys and you can even use them in your custom filters.

Indexed DB: To provide a key or not to provide a key

I am currently busy writing a little framework around the Indexed DB API to make it easier to use Indexed DB. One of the things I was struggling with was when the key parameter must or mustn’t be provided. So I tested every possible combinations.

For this I created 4 object stores with different combinations:

• Object store 1: KeyPath: “Id”, autoIncrement = true
• Object store 2: KeyPath: undefined, autoIncrement = true
• Object store 3: KeyPath: “Id”, autoIncrement = false
• Object store 4: KeyPath: undefined, autoIncrement = false

Next, I tried adding four different combinations, and here are the results.

Add/Put parameters		Object store configuration
Value	Key	Object store 1	Object store 2	Object store 3	Object store 4
{ Name: "test" }	undefined	successful	successful	failed	failed
{ Id: 1, Name: "test" }	undefined	successful	successful	successful	failed
{ Name: "test" }	1	failed	successful	failed	successful
{ Id: 1, Name: "test" }	1	failed	successful	failed	successful

 

Conclusion

1. When a KeyPath is defined, the key parameter must be undefined
2. When no KeyPath is defined and there is no autoIncrement, a key must be provided
3. When no KeyPath is defined and there is autoIncrement, a key can be provided
4. When a KeyPath is defined and there is no autoIncrement, an attribute with the name of the KeyPath must be present.

Indexed DB: Reading multiple records

In my previous post I have been talking about reading data. Today, I’ll be talking about reading multiple records at once. Here for we will use a cursor. A cursor is a transient mechanism used to iterate over multiple records in a database. The storage operations of the cursor can be used on the underlying index or an object store.

In the IDBObjectStore interface, we have the openCursor method to create a new cursor for retrieving data. In the IDBIndex interface, we have 2 ways to create a new cursor. These methods are openCursor to retrieve the values from the index and openKeyCursor to retrieve the keys. There are 2 optional parameters that can be provided when calling these methods. The first parameter is an IDBKeyRange, with this we will narrow the result by defining the bounds of the keys we want to retrieve. The second parameter is the direction the cursor must navigate trough the results.

IDBKeyrange

A key range is a continuous interval over some data type used for keys. A key range can have one of the following situations:

lower bounded: The keys must have a value smaller than the provided lower bound

upper bounded: The keys must have a value larger than the provided upper bound

lower and upper bounded: The keys must have a value between the lower and the upper bound

unbounded: All keys will be valid

Single value: The key must be the provide value

The upper and lower bound can be open, this means the value of the bound won’t be included, or closed, this means the value of the bound will be included.

More information about the IDBKeyRange interface can be found here. You will also find some more information about the methods to create a key rage. If you want to use an unbounded key range, you don’t need to provide a key range.

Retrieving data with a cursor

As for all actions preformed on the database a transaction is also needed in case of reading data with a cursor.

   1: var txn = dbconnection.transaction([“ObjectStoreName”]);

   2: var objectStore = txn.objectStore(“ObjectStoreName”);

   3: var cursorReq;

   4: // IE 10, Chrome and Firefox implementation

   5: if(window.msIndexedDB || window.mozIndexedDB || window.webkitIndexedDB){

   6:     cursorReq = store.openCursor();

   7: }

   8: // IE Indexed DB Prototype implementation

   9: else{

  10:     cursorReq = store.openCursor(IDBKeyRange.lowerBound(0));

  11: }

  12:  

  13: handleCursor(cursorReq, txn, success, error);

First things first, we define the cursor. One of the first thing you notice is that the IE Indexed DB prototype requires a key range. This means it won’t be possible to use the unbounded key range. Because I’m using an auto increment key for my object store, I can take use of the lowerbound method to create a key range from 0 to forever. After the cursor is defined, I use a method to handle the reading of the cursor.

   1: function handleCursor(cursorReq, txn, success, error){

   2:     cursorReq.onsuccess = function (event) {

   3:         if (event.result) {

   4:             var cur = event.result;

   5:             if (cur) {

   6:                 cursor_get_record(cur);

   7:             }

   8:         }

   9:         else if (cursorReq.result) {

  10:             var cur = cursorReq.result;

  11:             // Present the data

  12:             cur.continue();

  13:         }

  14:     cursorReq.onerror = error

  15: }

Again the IE Indexed DB prototype has a different implementation for handling a cursor. I’ll start with explaining the correct way. If the request object contains a result. If this is empty, you reached the end of the cursor. If the result is an object, the value of the current record in the cursor can be found in the value field. When you handled the value, you can navigate to the next record in the cursor by calling the continue function on the result object.

When working with the Indexed DB prototype, you have to make another approach. That’s why I created a recursive function for it. (cursor_get_record). The only parameter I need to pass is the result object I get out of the parameter that is provided when the call for the cursor was successful.

   1: function cursor_get_record(cur){  

   2:     cur.move(); 

   3:     if(cur.value)

   4:     {

   5:         // Present the data

   6:         cursor_get_record(cur);

   7:     }

   8: }

With the move method we fetch the next record from the cursor. If the cursor contains a value, you can handle it and afterwards you call the current method to fetch the next record of cursor.

Indexed DB: Reading data

Finally we get to the part where we can see the results of our Indexed DB tutorial. For the last few moths we have been structuring our database, adding data and now I’ll show how you can read data from an Indexed DB. For this we have 2 approaches. First one is retrieving a single record and the other way is retrieving a collection of records by a cursor.

In this post I’ll handle the retrieving of a single record. A post about retrieving data with a cursor will appear in the near future.

Retrieving data from an object store

In the IObjectStore interface we have an get method to retrieve data. This method must be provided with a key we want to search on. The key parameter can be a valid key or a key range. In case you don’t provide a valid key, a DATA_ERR exception will be thrown.

In the first situation a record of the object store will be retrieved where the given key is the same as the key of the key/value pair. In the other situation, the first record will be retrieved that matches one of the keys defined in the key range. If no record gets found, the result will be undefined. If you are using the Indexed DB prototype on IE 9, you’ll get an “Object with specified key not found.” error in stead of an result.

Time for some code:

When we want to read data, we need to create a transaction. This we do on the same way as for adding data. The only difference is we can use the IDBTransaction.READ_ONLY mode. This is the type that will be used as default when we create a transaction.

   1: var txn = dbconnection.transaction([“ObjectStoreName”]);

   2: var objectStore = txn.objectStore(“ObjectStoreName”);

The result of the get method is a IDBrequest object. When this is successful, the record is added to the result.

   1: var getReq = store.get(key);

   2: getReq.onsuccess = function(event) { 

   3:     var result;

   4:     // IE 9 implementation

   5:     if(event.result){

   6:         result = event.result;

   7:     }

   8:     // IE 10, Chrome and Firefox implementation

   9:     else if (getReq.result){

  10:         result = getReq.result;

  11:     }

  12:     // Code to present the result on your screen

  13: };

Retrieving data from an Index

In a previous post I have been talking about Indexes. Indexes make it possible to look up data from the object store by fields of the value object. By example: we have an object store person. In this object store we store persons. A person object has a first name and a last name field. By creating an index with a key path “last name”, we make it possible to retrieve data from the object store by providing the last name of the person.

In the IDBIndex interface we have the get method to retrieve a record from the object store, and the GetKey method to retrieve the key of the record from the object store. In both cases a key must be provided. This key parameter can be a valid key or a key range. For the rest every thing works the same as for retrieving data from an object store.

One think you need to keep in mind. In the Indexed DB prototype on IE9 the get and getKey method work different. In this case the getKey method retrieves the data and the get method the key of the record.

   1: var txn = dbconnection.transaction([“ObjectStoreName”]);

   2: var objectStore = txn.objectStore(“ObjectStoreName”);

   3: var index = store.index(“IndexName”);

   4: var getReq = index.getKey(key);

   5:  

   6: getReq.onsuccess = function(event) { 

   7:     var result;

   8:     if(event.result){

   9:         result = event.result;

  10:     }

  11:     else{

  12:         result = getReq.result

  13:     }

  14:     // Code to present the result on your screen

  15: };