pages

Server-Side Paged Lists in Knockout

After my last post on client-side paging using Knockout I had a few people comment that it would be useful if something reusable could handle server-side paging. It’s not quite what I needed but it sounded useful so I thought I’d see what I could come up with…

Strategising the Server

Server-side paging poses a slightly different problem to paging on the client-side as it requires (obviously) some cooperation from the server. Given that the server might be implemented in any of a hundred different ways I decided to use a strategy pattern to actually load the data:

var ViewModel = function() {
    this.pagedList = ko.pagedList({
        loadPage: function(pageIndex, pageSize) {
            //load data here
        }
    });
};

The requirements of the strategy are:

  1. Accept a pageIndex and pageSize to be loaded
  2. Return a jQuery.Deferred instance (such as that returned by $.ajax)
  3. Resolve the deferred object with a value that contains an array rows property

Assuming that the server response needs no modification (i.e. already has an array rows property) then a paged list can be created as below:

var ViewModel = function() {
    this.pagedList = ko.pagedList({
        loadPage: function(pageIndex, pageSize) {
            return $.getJson("/getPage?index=" + pageIndex + "&size=" + pageSize);
        }
    });
};

Getting the Total Number of Rows

When writing this I was keen to avoid enforcing anything on the server implementation that wasn’t absolutely necessary, and one of the things I decided wasn’t needed was the total number of available rows. As the client side component cannot always know how many total rows are available, I decided that by default I would simply set the totalRows property to -1 and the pagedList will function perfectly without ever knowing how many rows are actually available – with the following limitations:

  • pageCount and totalRows will always return -1
  • nextPage will never prevent the user from moving to the next page, even if that page will be empty

That being said, I also wanted to support those scenarios where the total number of rows is available, so totalRows can be set in one of two ways:

Manually Set totalRows

As with any Knockout observable property the totalRows value can be set externally, and the pagedList will use whatever value is set to calculate pageCount and prevent moving beyond the final page using nextPage.

var vm = new ViewModel();
vm.pagedList.totalRows(100); //specify 100 rows total

Include totalRows in Server Response

The minimum requirement of the data returned from the page loading strategy is that it requires an array rows property, but if it also includes an integer totalRows value then this will automatically set the observable value on the pagedList.

this.pagedList = ko.pagedList({
    loadPage: function(pageIndex, pageSize) {
        var defer = $.Deferred();

        //this is an example of the expected response object from the server
        defer.resolve({
            rows: [1, 2, 3],
            totalRows: 100 //specify the total number of rows in the response
        });

        return defer;
    }
});

In this way we can either specify the total number of rows on the client side, on the server side, or not at all – and still use the pagedList in the same way.

Pre-Loading the First page

When the page containing the pagedList is being generated on the server it is preferable that the first page of data already be loaded, rather than waiting on an unnecessary post back to the server to populate the list.

With that scenario in mind, I added the ability to specify the first page of data as another option on the constructor.

this.pagedList = ko.pagedList({
    loadPage: function(pageIndex, pageSize) {
        //...
    },
    firstPage: {
        rows: [1,2,3],
        totalRows: 100
    }
});

If this option is specified then the pagedList will not attempt to load the first page of data and will instead render the data specified in the option.

Note: this is also another mechanism by which the totalRows property can be pre-populated on the client side.

Mapping Data to ViewModels

In most cases the server will be returning something more complicated than the numeric values in the examples above, and in those cases it is common to want to wrap the returned object in a view model of it’s own.

To support this functionality I included a map constructor parameter that will be invoked on each of the rows returned from the page load strategy.

this.pagedList = ko.pagedList({
    //...
    map: function(item) {
        return ko.mapping.fromJS(item);
    }
});

Note: this example makes use of the Knockout mapping plugin to generate a view model.

Bindable Properties

Much of the behaviour is similar to the pagedObservableArray as it was described in my original post, with the most significant difference being that the returned object can be used directly – there is no need to bind to a page property as there was before. It also exposes the following properties:

  • pageSize – An observable instance containing the number of items per page
  • pageIndex – An observable instance containing the zero-based current page index
  • totalRows – An observable instance containing the total number of rows that are assumed to be available (or -1 if this has not been determined)
  • pageCount – A computed observable that returns the number of pages of data available (or -1 if totalRows is unknown)
  • previousPage – A command that moves to the previous page, if possible
  • nextPage – A command that moves to the next page, if possible
  • loadPage – A command that loads the page data at the index specified as the first parameter. The loadPage.isRunning observable should be used for all loading notifications

One point to note is that this implementation does not expose an observableArray containing the data; instead it exposes a regular observable with a value that is an array. This may seem a little strange but I felt that the additional functions and properties on the observable array do not really make sense when talking about a collection of server-specified data.

Source Code & Samples

The source code and unit tests are available on GitHub, and a working sample is available on jsFiddle.

If anyone else has any other suggestions or requests just let me know in the comments!

Advertisements

Command Pattern v2 using Knockout

Update: this feature is now available as part of the ko.plus library available on GitHub and NuGet!


In an earlier post I outlined a basic implementation of a command pattern using Knockout that tracks the progress of an asynchronous operation using the jQuery.Deferred object.

Having used it for a couple of weeks I found that there were a few things that could do with improving, so here we go with Command v2!.

No more .execute()

The first version of the command would wrap the operation in an object, meaning that whenever you wanted to bind to or just run the action you had to call a .execute method on the new object:

//old code
var viewModel = {
    doSomething: new Command(function() {
        //invoke async operation        
    });
};

//invoke the command
viewModel.doSomething.execute();​

Whilst using the command I kept forgetting to add the .execute (particularly to bindings) and this quickly became annoying, so I changed the constructor function to return the execution function instead of a new object in a similar manner to ko.observable.

Note: having previously been returning this I had to ditch the traditional constructor for a factory method that is a bit more in line with the Knockout syntax.

ko.command = Utils.command = function (options) {
    //...

    //execute function (and return object
    var _execute = function () {
       //...
    },
    
    //public properties now attached to _execute instead of this
    _execute.isRunning            = _isRunning;
    _execute.done                 = _done;
    _execute.fail                 = _fail;
    _execute.always               = _always;

    return _execute;
};

After these changes the invocation of the command is much simpler:

var viewModel = {
    doSomething: ko.command(function() {
        //invoke async operation 
    });
};

//invoke the command
viewModel.doSomething();

//get the status of the operation
viewModel.doSomething.isRunning(); 

//attach a success handler
viewModel.doSomething.done(function() {
    alert("Finished");
});​

That’s much cleaner, and the helper observables and functions such as isRunning and the callback methods still function as before.

Support for canExecute

The command pattern implementation in .NET (the original inspiration for writing this) includes a CanExecute method that determines whether or not the command is currently in a suitable state to execute. For example, if the command is going to submit a form, the CanExecute implementation might check that the form is valid prior to submission.

For my JavaScript command I wanted to be able to specify a custom function to determine whether or not the command can execute, as well as always preventing execution whilst the command is running asynchronously. To do this I made use of Knockout’s computed observables to implement a canExecute property:

ko.command = Utils.command = function (options) {
    //factory method to create a $.Deferred that is already completed
    _instantDeferred = function(resolve, returnValue) {
       //
    },

    //execute function
    var _execute = function () {
        //check if we are able to execute
        if (!_canExecute()) {
            //dont attach any global handlers
            return _instantDeferred(false).promise();
        }

        //...
    },
    
    //dummy to allow us to force a re-evaluation of the computed _canExecute observable
    _forceRefreshCanExecute = ko.observable(), 

    //canExecute flag observable
    _canExecute = ko.computed(function() {
        _forceRefreshCanExecute(); //get and ignore the value

        //can execute if we're not running, and either there's no canExecute
        //function specified or it returns true
        return !_isRunning() &&
            (typeof options.canExecute === "undefined" ||
            options.canExecute.call(_execute));
    }, _execute),
    
    //invalidate canExecute
    _canExecuteHasMutated = function() {
        _forceRefreshCanExecute.notifySubscribers();
    };

    //public properties
    _execute.canExecute           = _canExecute;
    _execute.canExecuteHasMutated = _canExecuteHasMutated;

    return _execute;
};

Things of interest here:

  • If the _execute method cannot execute, it returns a $.Deferred object that has already been rejected so that any calling class that attaches a done/fail handler will not throw exceptions and can at least sensibly react
  • We are creating a dummy observable (_forceRefreshCanExecute) that is used by the canExecuteHasMutated function to force the computed canExecute observable to refresh (as described here)

Synchronous Execution

The first version relied on the action returning a promise object representing the status of the background operation, but after implementing the canExecute support I started to see a use case for commands that do not do anything in the background and simply execute synchronously.

var ExampleViewModel = function(externalValidationDefinition) {
    var _self = this;
    this.listOfItems = ko.observableArray();
    this.clearList = ko.command({
        action: function() {
            _self.listOfItems.removeAll();
        },
        canExecute: function() {
            return _self.listOfItems().length > 0;
        }
    });
};

To accommodate this type of usage I have updated the behaviour of the command so that when the action returns anything other than a promise it wraps that result in an immediately-completed jQuery.Deferred instance and then executes any registered handlers.

Other Minor Changes

errorMessage Removed

I realised that the error handling in version 1 was far too dependent on the signature of the error handler – what works for a failed AJAX request will not work for a failed HTML5 SQL request etc. – so I elected to remove the built-in error message recording and leave that down to the user.

Better Error Handling

As part of the support for synchronous code I also added improved error handling so that any error thrown directly from the action will be passed to the fail handlers.

Source

As ever, the code and unit tests are available on GitHub so help yourselves.

Forcing Computed Observables to Refresh in Knockout

Computed Observables in Knockout are a useful way of calculating values based on other observables:

var ExampleViewModel = function() {
    this.name = ko.observable();
    this.isNameValid = ko.computed(function() {
        return this.name().length > 0;
    });
};​

One of the great features here is that when the name property changes the Knockout framework will automatically re-evaluate the computed observable and notify anything that is bound to that value. This works by automatically subscribing to the change notifications from the name observable when the value if first acquired.

Unfortunately this does not work when the value of the computed observable is dependent on a non-observable value:

var ExampleViewModel = function(externalValidationDefinition) {
    this.name = ko.observable();
    this.isNameValid = ko.computed(function() {
        return this.name().length > externalValidationDefinition.minLength;
    });
};

var validationDefinition = {
    minLength: 5;        
};

var vm = new ExampleViewModel(validationDefinition);

//change the validation definition...
validationDefinition.minLength = 10;
//...but the computed observable is not re-evaluated

In this situation we need to manually force the re-evaluation of the computed observable, but there is no native way of doing so.

To get around this we can add a dummy observable to the view model, then retrieve and ignore its value from within the computed observable. This will cause the Knockout framework to re-evaluate the computed observable whenever the dummy observable changes:

var ExampleViewModel = function(externalValidationDefinition) {
    var _dummyObservable = ko.observable();

    this.name = ko.observable();
    this.isNameValid = ko.computed(function() {
        _dummyObservable(); //retrieve and ignore the value
        return this.name().length > externalValidationDefinition.minLength;
    });
};

Now we can invalidate the computed observable just by invalidating the value of the dummy one:

var ExampleViewModel = function(externalValidationDefinition) {
    var _dummyObservable = ko.observable();

    this.name = ko.observable();
    this.isNameValid = ko.computed(function() {
        _dummyObservable(); //retrieve and ignore the value
        return this.name().length > externalValidationDefinition.minLength;
    });
    this.invalidateIsNameValid = function() {
        _dummyObservable.notifySubscribers(); //fake a change notification
    };
};

var validationDefinition = {
    minLength: 5;
};

var vm = new ExampleViewModel(validationDefinition);

//change the validation definition...
validationDefinition.minLength = 10;
//...and manually invalidate
vm.invalidateIsNameValid();​

Now we can manually force the computed observable to be updated when we know that an external factor has changed. If I get the time I may convert this into an extender or helper for reusability later…

Paging Lists with Knockout

As covered by this StackOverflow question, Knockout can get a little bit slow when trying to render large amounts of data.

This has been particularly noticeable in a recent project that has to run on older iPads, so I took the time to put together a simple paging solution. You can see the source code (and tests) here or you can see a running example here.

To make use of this object, create an instance of Utils.PagedObservableArray as below:

var ViewModel = function(data) {
    this.pagedList = new Utils.PagedObservableArray({
        data: data,
        pageSize: 3
    });
};

You can then bind to the exposed properties using some simple HTML:

<div data-bind="with: pagedList">
    <div>
        <a href="#" data-bind="click: previousPage">Previous Page</a>
        <span data-bind="text: 'Page ' + (pageIndex() + 1) + ' of ' + pageCount()"></span>
        <a href="#" data-bind="click: nextPage">Next Page</a>
    </div>
    <ul data-bind="foreach: page">
        <li data-bind="text: $data"></li>
    </ul>
</div>

The new instance wraps the existing observableArray object from the Knockout library and exposes a number of properties to support paging:

  • allData – An observableArray instance exposing the entire data set. This should be used to populate and to access the source data
  • pageSize – An observable instance containing the number of items per page
  • pageIndex – An observable instance containing the zero-based current page index
  • pageCount – A computed observable that returns the number of pages of data available
  • page – An observableArray instance that contains the current page of data
  • previousPage – A function that moves to the previous page, if possible
  • nextPage – A function that moves to the next page, if possible

The final implementation is:

(function (Utils, ko) {
	Utils.PagedObservableArray = function (options) {
		options = options || {};
		if ($.isArray(options))
			options = { data: options };
		var 
		//the complete data collection
        _allData = ko.observableArray(options.data || []),

		//the size of the pages to display
        _pageSize = ko.observable(options.pageSize || 10),

		//the index of the current page
        _pageIndex = ko.observable(0),

		//the current page data
        _page = ko.computed(function () {
        	var pageSize = _pageSize(),
                pageIndex = _pageIndex(),
                startIndex = pageSize * pageIndex,
                endIndex = pageSize * (pageIndex + 1);

        	return _allData().slice(startIndex, endIndex);
        }, this),

		//the number of pages
        _pageCount = ko.computed(function () {
        	return Math.ceil(_allData().length / _pageSize()) || 1;
        }),

		//move to the next page
        _nextPage = function () {
        	if (_pageIndex() < (_pageCount() - 1))
        		_pageIndex(_pageIndex() + 1);
        },

		//move to the previous page
        _previousPage = function () {
        	if (_pageIndex() > 0)
        		_pageIndex(_pageIndex() - 1);
        };

		//reset page index when page size changes
		_pageSize.subscribe(function () { _pageIndex(0); });
		_allData.subscribe(function () { _pageIndex(0); });

		//public members
		this.allData = _allData;
		this.pageSize = _pageSize;
		this.pageIndex = _pageIndex;
		this.page = _page;
		this.pageCount = _pageCount;
		this.nextPage = _nextPage;
		this.previousPage = _previousPage;
	};
})(Utils, ko);

The implementation itself is not doing anything particularly clever but there are a couple of areas to note:

  • By using jQuery.isArray we can determine whether the parameter passed in is an array or just another object. If it is an array we just want to use it as the source data to save callers from wrapping their data in an options object.
  • Using Array.slice on the return value of allData in the page computed observable means that we can use the exposed allData collection as you would any other observableArray and the computed observable gets update notifications for free.
  • We are using the subscribe method directly on both pageSize and allData to reset the current page index to zero whenever either value changes
  • We use Math.ceil to get a whole number of pages for pageCount, with a special case to return a page count of 1 where there are no items in the source data