Indexed DB: The basics

The Indexed DB API has the purpose to provide storage for large numbers of objects. This is necessary to satisfy offline data requirements. The web storage API is useful for storing data as key-value pairs, but has some limitations. It only has 5MB of storage space available, it does not provide in-order retrieval of keys, efficient searching over values, or storage of duplicate values for a key.

The Indexed DB specification provides an advanced key-value data management. It does this by using a transactional database to store their keys and corresponding values (more than one value for 1 key is allowed) and providing a means of traversing keys in a deterministic order.

The Indexed DB specification declares 2 different API’s. You have an asynchronous that is currently implemented by the most modern browsers (IE 10, Chrome 11.0, Firefox 4), and you have an synchronous API. This last one isn’t yet implemented  in any browser. The synchronous Indexed DB API methods provide a blocking access pattern to Indexed DB databases. Since they block the calling thread they are only available from workers. But first some definitions

Database

For every origin you can create an infinite number of databases. The only thing you need to create a database is an unique name. A database also has a version, this version will be used to determine the structure of the database. When a database is created for the first time, the version will be an empty string. Each database can only have one version at a time, this means the database can’t exist in multiple versions at once.

Object store

A database compromises one or more object stores. These object stores hold the data that is stored into the database. This data is stored as a list of records. Each record exists of a key and a value. The keys are sorted in ascending order by default and a key can never be exist multiple times within one object store.

Every object store has a name that is unique within the database. Optionally it has an key generator and a key path. A key generator generates a monotonically increasing numbers every time a key is needed. When a key path is given, the key path will be used as key. The last possibility to provide a key, is by explicitly specifying it when storing the value.

The set of object stores can be changed, but it can only change by using the Version_change transaction. This transaction will change the version of the database and change the set of object stores as you defined.

key path

A key path is a DOMstring and defines how we can extract the key from the value. For example if you are using JSON, the DOMstring will be the name of your property.

Keys

To enable the most efficiently manner to retrieve records stored in the database, each record is organized by it’s key. A valid key can be one of the following types:

• Array of JS objects
• A DOMstring
• A data
• and a float

Index

An index allows looking up records in a object store using properties of the values in the object stores records. It is a specialized key-value storage that has a referenced object store. The index has a list of records which hold the data stored in the index. The records in an index are automatically populated whenever records in the referenced object store are added, update or deleted. It’s possible that multiple indexes are referencing to the same object store, so when the object store changes all the indexes get updated.

The keys for the index are derived from the referenced object store's values using a key path. The index also contains a unique flag. When it’s true it will enforce that the key will be unique over all the records in the object store.

Transactions

A transaction will be used to interact with the database, it represents an atomic and durable set of data access and mutation operations. All data that is read or written to the database will happen using an transaction. To create a transaction, a connection is needed. Each transaction has a mode which declares what type of interactions are allowed within the transaction. This mode is set when creating the transaction and will remain the same for the lifetime of the transaction. It also has as scope, this declares the object stores that can be affected within the transaction. An active flag determines if new request can be added. And with the request list you get an overview of all the requests that have been placed against the transaction.

The available modes are:

• READ_ONLY
• READ_WRITE
• VERSION_CHANGE

Request

Every read or write operation on the database is done by a request. A request exists out of a flag done which indicates if the request is completed and  a source object. If the request is completed the result or error code become accessible.

Cursor

A cursor is a mechanism to iterate over multiple records in the database. It compromises a range of records in either an index or a object store. The source of the cursor contains the index or object store we want to iterate. The cursor also has a position which indicates the object it is currently working with and a direction where to the cursor is moving. A position can’t be seen as an index, but rather as a key of the previously returned record. The key and value property of the cursor contain the key and value the cursor is currently holding.

This was a brief introduction to the Indexed DB API, more information about Indexed DB can be found here. In a future post I will handle about creating a database and his structure.

New features of IE10

With the release of the Windows 8 Developer preview on Wednesday at the BUILD conference, the third preview of IE10 was released. I had never expected that this would bring that many new features. Some of the features were already available at the html5labs, but needed sever installations to get it work. But from now all these features are released in the new preview of IE10. Even features who weren't available at html5labs are added.

From now it will be possible to build web applications that will work online as well offline. The API for this is the HTML Application Cache. In a manifest file you describe all the resources (HTML files, JS files, CSS files, Images, …) you need to let your application work offline. For storing your data offline, we now have 2 possibilities. We have the Local storage we can use to save key/value pairs. This is handy if we save small amounts of data and do not need a structured way to be saved or need to query on data in the value. The second possibility is the use of IndexedDB. This is an offline database that runs in the client. You can’t use SQL to query on it, but it has a set of interfaces that provide methods to create, read, update and delete your data and structure.

Another feature that was available at html5labs was web sockets. Web sockets enable you to use TCP to communicate between your browser and the webserver. On this way we can save a lot of traffic when build applications who need frequent polling to get there data. When using HTTP we are always sending an HTTP header. In a web socket message we send only add 0x00 at the begin of the message and 0xFF at the end.

With the history API we will be able to manipulate the URL without having a page refresh or navigation. This way we can use AJAX to give the user a fluent way to work with your application (without page refresh) and adjust the URL at the same time. This enables the user to move back and forward in your web application who uses AJAX.

To improve the performance of your web applications, IE added the possibility to run scripts Asynchronously and the use of Web workers to make your web application multithreaded in your browser.

The File API provides the web developers a secure way to access files on the local machine. This way the user can manipulate local files without having to upload it to the server.

Other improvements are:

• Rich Visual Effects: CSS Text Shadow, CSS 3D Transforms, CSS3 Transitions and Animations, CSS3 Gradient, SVG Filter Effects
• Sophisticated Page Layouts: CSS3 for publication quality page layouts and application UI (CSS3 grid, flexbox, multi-column, positioned floats, regions, and hyphenation), HTML5 Forms, input controls, and validation
• Enhanced Web Programming Model: Better offline applications through local storage with IndexedDB and the HTML5 Application Cache; Web Sockets, HTML5 History, Async scripts, HTML5 File APIs, HTML5 Drag-drop, HTML5 Sandboxing, Web workers, ES5 Strict mode support.

Getting started with IndexedDB on IE

I have been playing with HTML5 for a while, and more and more I’m seeing the advantage of building applications in HTML 5. In particular applications that will run online and offline as well. One of the things we need for this is storage on the client side. For now most browsers only support WebStorage for this. The disadvantage of Webstorage is the maximum of 5mb of data and every thing is stored in KeyValue pairs, which make searching in the data harder.

Some browsers are now experimenting with the IndexedDB spec. A description of the spec can be found here. In FireFox en Chrome the IndexedDB API is all ready build-in. Not in the window.IndexedDB, but in window.mozIndexedDB for FireFox and window.webkitIndexedDB for Chrome. For internet explorer, you can use an experimental API that you can find on html5labs.

To get started with IndexedDB for Firefox and Chrome we don’t need to install something, to get started with IndexedDB for Internet explorer, we need to install the IndexedDB prototype. The files we need to install can be found here. Once the file is downloaded, we unpack the zip-file.

The first thing we need to do is run the vcredist_x86.exe which can be found in the map we just unpacked. Once that is installed, we need to install the sqlcejse40.dll:

1. Open the command prompt as administrator
2. Navigate with the cd command to the unpacked zip-file
3. Run the following command:
   regsvr32.exe sqlcejse40.dll

Once we pasted these steps, we should be able to open and use the demo included in the zip-file. But beware, you need to run you Internet Explorer with administrator right, otherwise you will get the following error in the error textbox:

6 - Error 0x80040E09, minor 25039 (0x000061CF).

Now when everything is properly installed, we can start to develop. But, this will be content of a next blogpost.

Configuring webserver for html5 video

When u are using HTML 5.0 video on your website, you need to remember that your webserver is configured to send the correct MIME–type with your file. If you don’t the browsers won’t recognize it and the video won’t play. You will see an X appearing on your video, and the video will not start playing when you click on it.

If u are using an Apache webserver, you can use the AddType directive in your side-wide httpd.conf or in the .htaccess file that is located in your directory where your videos are stored.

The following line should be added in that case:

AddType video/ogg .ogv
AddType video/mp4 .mp4
AddType video/webm .webm

When u are using an IIS server there are 2 possibilities. You can configure it on your IIS server, or you can add the MIME–types in your Web.config.

When you want to configure the MIME–types on your IIS server there are some differences between the versions.

Configuring MIME-Type for IIS 6.0
Configureer MIME-type for IIS 7.0

The other possibility is using your Web.config. This is ideal for developers who are use IIS Express and don’t have a full instance of IIS running. But beware, this is only supported when you are using IIS 7.0 or higher. The following section needs to be added in your configuration file:

<configuration>
  <system.webServer>
    <staticContent>
      <mimeMap fileExtension=".mp4" mimeType="video/mp4" />
      <mimeMap fileExtension=".webm" mimeType="video/webm" />
      <mimeMap fileExtension=".ogv" mimeType="video/ogv" />
    </staticContent>
  </system.webServer>
</configuration>