Linq2IndexedDB: Debugging IndexedDB

In my latest version of the Linq2IndexedDB 1.0.7, I added the viewer object. This object allows you to inspect your IndexedDB database and gives you an overview of all the object stores and indexes that are present. It even gives you details about how they are configured and what data they contain. Also using this, you can take a look at your database in various debugging tools like:

• Google Chrome Developer tools
• Firefox Developer tools
• IE Developer tools
• Visual Studio

The viewer object contains 4 properties:

1. configuration: In here you will find the way the Linq2IndexedDB framework is configured. (The same properties as you can define in the configuration object are present +  a property indicating if framework is running in auto create mode.)
2. name: The name of the IndexedDB database
3. objectStores: A collection of all the object stores that are present in the database
4. version: The current version of the database

Inside an object store object, we can see the following:

1. autoIncrement: indicates if the keys for the object store are generated by the IndexedDB API
2. data: A collection of all the data available in the object store. These data objects are shown as key/value pair objects.
3. indexes: A collection of all the indexes available on the object store.
4. keyPath: The property inside the object that keeps the key
5. name: The name of the object store

The last level we can take a look at, are the indexes. The following information will be provided:

1. data: A collection of all the data available in the object store. These data objects are shown as key/value pair objects.
2. keyPath: The property inside the object that keeps the key
3. multiEntry: Determines if a key is only once defined in the index (which means the value keeps an array of all the values in the object store who match) or the key can be present multiple times (once for every value that matches in the object store.)
4. name: The name of the index

conclusion

I hope this debugging information can help you speed up your software development and finding bugs while working with the IndexedDB API. The latest version of the framework can be found on codeplex, nuget and MyGet.

Update 28/07/2012

I added a new version of the framework, because there were some little bugs with the viewer. The following things change:

• The viewer no longer blocks the deletion or upgrade of the database
• The viewer now gets updated when the structure of the database changes
• The viewer is default disabled for performance issues. If you want to enable it, add true as 3th argument on the function to create a linq2indexeddb function

   1: var db = $.linq2indexedDB("test", null, true);

Linq2IndexedDB: Custom filters

Since the 1.0.5 version of the Linq2IndexedDB framework, I added support for custom filters. And since the 1.0.6 version (released last week) you can add a function to the where clause to filter your data. These functionalities are provided by the framework, because the IndexedDB API only provides a handful filter possibilities. Also you can only use one filter when retrieving data.

Create a custom filter

To create a custom filter, the Linq2IndexedDB framework has an addFilter method. This method can be found in the linq namespace and accepts 3 arguments.

• The first argument is a name for the filter, this name will be used when you want to use your filter. Make sure that this name is unique over all the filters, if not an exception will be thrown.
• The second argument is a function that will be used to determine if the data is valid or not. When this function gets called 2 arguments will be passed. The first one is the object to validated. The second one is filter metadata. For example this object contains the name of the property you need to filter on, or an additional value provided, … The result of this function must be a Boolean value telling if the provided object is valid or not.
• The last argument is also a function and will be used to retrieve additional filter metadata. This function must return a new function with optionally arguments to retrieve the additional in formation. For example this information can be one or more values to use in the IsValid function. When called 3 arguments are passed to this function. The first one is a callback function that needs to be the return value of the function to retrieve the information. The second argument is the queryBuilder object that needs to be passed to the callback method. The third argument is the filterMetaData object. The additional information you retrieve, needs to be added to this object so you can use it later on in the IsValid method. After this is done, this object must also be provided as argument to the callback method.

In the code below I added an example of an equals filter.

   1: linq2indexedDB.prototype.linq.addFilter("equals", function (data, filter) {

   2:     return data[filter.propertyName] == filter.value;

   3: }

   4: , function (callback, queryBuilder, filterMetaData) {

   5:     return function (value) {

   6:         filterMetaData.value = value

   7:         return callback(queryBuilder, filterMetaData);

   8:     }

   9: });

When all this is done, you can start using this filter just by calling it after the where, or and and method call.

   1: db.from("objectStore").where("property").equals("value").select()

Anonymous filters

An other way to add a custom filter is by adding a callback function to the where, or or and method. When this callback function gets called, the object you need to validate is passed as an argument. The result of the callback function must be a Boolean value determining if the object is valid or not

   1: db.linq.from("objectStore").where(function (data) {

   2:     return data.Age > 3;

   3: }).and(function (data) {

   4:     return data.Age < 10;

   5: }).select();

conclusion

In the Linq2IndexedDB framework, you now have 2 ways to add custom filters. In the case you use the addFilter method, you can add a custom filter that can be reused and will appear in intellisense so you can easily call it. The other way provides an easy only once use of a filter. In both cases you are now flexible to provide your own filtering.

JSON: serialize and deserialize functions in JavaScript

In my Linq2IndexedDB project, I take advantage of the web workers to do all the filtering that the IndexedDB API doesn’t allow (multiple filters, like, inArray, …). For all these filters I have an “isValid” method which determines if the data satisfies the condition. So when I want to use these methods in my background worker, I need to make sure I can call them. For that I have 3 possibilities

1. Add a copy of all the filters in my background worker file
2. Include the file where my filters are defined
3. Serialize and deserialize the “isValid” functions

The first one wasn’t an option for me, because i wanted to keep all filter logic at one place. The second one was an option, but I didn’t want to use multiple JavaScript files for my library. And an other reason why I don’t like the first 2 is because developers would have to change my library if they want to add additional filters. So this left me only with the third possibility.

For this I’ve done some research. I already knew that you could call .toString on a function and this would result into a string representation of the function. And with the Function.call method I would be able to call the function in my background worker. But I didn’t wanted a solution that had to call the Function.call method in the background worker, I wanted to use the .isValid method that was provided in the filter objects. So I dug into the JSON API and found the following solution.

Serialize Objects

The JSON.stringify (the method that turns JavaScript objects into JSON text) accepts 2 arguments. The first argument is the object you want to turn into a JSON text and the second argument accepts a replacer function. This function gets called for every value in the object structure (even for properties in child objects) and accepts a key (name of the property) and a value (the value of the property) argument. The return value of the object is the object that will be stringified. By using this function, I can return the string representation of the function (in case of function) and return the value in the other cases.

   1: JSON.stringify(filters, function (key, value) {

   2:     if (typeof value === 'function') {

   3:         return value.toString();

   4:     }

   5:     return value;

   6: });

deserialize objects

The next thing we want to do, is deserialize the function again. Like the stringify method, the parse method (returns a JSON string into a JavaScript objects) also a callback (reviver) method as second argument. This method also gets called for every key and value for every level of the result. In this callback you can reform generic objects into instances of pseudo classes, strings into dates, strings into functions, …

   1: JSON.parse(filtersString, function (key, value) {

   2:     // reform your objects here

   3: });

Once I have the string representation of the function, I can start rebuilding that function. This is done by creating a new Function Object. The constructor of the Function objects accepts 2 arguments, a list of arguments for the function you want to create and the string representation of the body of the function you want to create.

   1: new Function(arguments, functionBody);

So if we put those 2 together we get the following:

   1: JSON.parse(filtersString, function (key, value) {

   2:     if (value 

   3:         && typeof value === "string" 

   4:         && value.substr(0,8) == "function") {

   5:         var startBody = value.indexOf('{') + 1;

   6:         var endBody = value.lastIndexOf('}');

   7:         var startArgs = value.indexOf('(') + 1;

   8:         var endArgs = value.indexOf(')');

   9:  

  10:         return new Function(value.substring(startArgs, endArgs)

  11:                           , value.substring(startBody, endBody));

  12:     }

  13:     return value;

  14: });

Detecting if the value is a function, is done by checking if the value is a string and starts with the word ‘function’. If this is the case, we will determine the arguments and the function body so we can pass it to the Function constructor. Retrieving the arguments of the new function is done by taking the string value between the first ‘(‘ and ‘)’. Retrieving the function body is done by getting the string value between the first ‘{‘ and the last ‘}’. Passing these 2 values to the constructor of the function will create a new function. This is the value that gets returned in case of a function. In all other cases we just return the value.

Conclusion

By using the stringify and parse method of the JSON API you can provide a generic way to serialize and deserialize your objects. Serializing functions can easily be done by calling the toString method on the function and deserializing a function can be done by using the Function constructor which is present in the JavaScript language.