Simplify Long Operation Handlings

{{Template:Smalltalk_Author|
|author=Robert Wenzel, Engineer, Potix Corporation
|date=January 22, 2015
|version=ZK 7.0.4
}}

Special thanks go to Matthias Thieroff (T-Systems on site services GmbH) for the idea and co-development of a similar helper class (which was the basis for this article).

= Introduction =

Whenever a server side operation takes a longer time than the user expects ( > 2 seconds) I consider it a "Long Operation" that requires some feedback to either keep the user waiting, while giving status updates, and ideally let the user still interact with the UI while the operation is running. That requires to run the task in the background on the server without blocking the UI, or blocking the UI (or parts of it) and inform the user about the estimated duration, or the percentage of completeness as a few examples.

Because Long Operations handling is an important topic ZK already supports [http://books.zkoss.org/wiki/ZK_Developer%27s_Reference/UI_Patterns/Long_Operations technical solutions] for this task. 

However some questions are often raised: 

* How to integrate them in your already complex UI?
* How to separate the "ugly" technical details from your ViewModels/Composers?
* How to cancel a task? 
* How to run parallel tasks?
* How to do the same thing in MVVM/MVC?

This Small Talk demonstrates a simple object-oriented approach/idea leveraging the existing ZK ServerPush mechanism and the Java's Thread api.
It can be used with both MVC and MVVM (while the examples here use MVVM it should be obvious that the same code can be implemented in an MVC event listener).

= Long Operations made simple =

The 
[https://github.com/cor3000/zk-long-operations/blob/master/src/main/java/zk/example/longoperations/LongOperation.java LongOperation] class 
provides a basic abstraction over the required ZK and Java Threading details to handle a (cancellable) long running operation using ServerPush to give feedback to the user - during and after the operation. The class will take care of starting/stopping ServerPush to save resources when no LongOperation is active. (You should watch your Network tab before, during and after the long operations, and notice that the ServerPush is only activated when required).

It provides a set of methods that can be called/overridden to implement your Long Operation requirements.

Public control methods (to start/cancel)
* '''start()''' - will launch an asynchronous Thread and call the life cycle methods below
* '''cancel()''' - request to cancel the task from outside


Overridable life cycle callbacks
# '''execute()''' - your long operation implementation (runs in a separate thread)
# either one of the methods below is called for the 3 possible outcomes
#* '''onFinish()''' - the operation terminated successfully (perform your final UI updates here)
#* '''onCancel()''' - the operation was cancelled (by an InterruptedException)
#* '''onException(Exception e)''' - handle any unexpected exception
# '''onCleanup()''' - always called after any outcome (do your general cleanup work here)


Protected helper methods
* '''activate()''' - start UI updates during the task 
* '''deactivate()''' - finish UI updates during the task and send them to the browser
Activate()/deactivate() are called automatically around these 4 callback methods (onFinish, onCancel, onException, onCleanup).

You can call these 2 methods anytime during your long operation to perform intermediate updates (e.g. updating status information for the current task).
Another use case is to create additional life cycle methods when [[#Extending LongOperation | Creating a custom LongOperation]].

== Minimal Usage ==

The minimal usage requires to implement '''execute()''' and call '''start()'''.

<source lang="java" high="5, 8">
@Command
public void startLongOperation() {
	new LongOperation() {
		@Override
		protected void execute() throws InterruptedException {
			Thread.sleep(5000); //you'll do the heavy work here instead of sleeping
		}
	}.start();
	Clients.showNotification("background task started");
}
</source>

This does not do much, it simply waits for 5 seconds without blocking the UI, and then silently finishes.

== UI-feedback at finish ==

Let's give the user some feedback that the task is done using '''onFinish()'''.

<source lang="java"  high="11">
@Command
public void startLongOperation() {
	new LongOperation() {
		
		@Override
		protected void execute() throws InterruptedException {
			Thread.sleep(5000); //you'll do the heavy work here instead of sleeping
		}

		@Override
		protected void onFinish() {
			Clients.showNotification("done sleeping for 5 seconds");
		};
	}.start();
		
	Clients.showNotification("starting, you'll be notified when done.");
}
</source>

This time the operation will display an information message when the background task is done.

== UI-feedback in the middle of the operation ==

To update the UI between your steps simply call '''activate()''' and '''deactivate()''' around your UI updates.

<source lang="java"  high="5, 7">
	@Override
	protected void execute() throws InterruptedException {
		doFirstHalf();

		activate();
		Clients.showNotification("50% Done"); // any UI updates in here
		deactivate();

		doSecondHalf();
	}
</source>

You can update the UI several times here an example with more steps:

<source lang="java" high="3, 5, 7, 9, 13">
	@Override
	protected void execute() throws InterruptedException {
		showStatus("step 1");
		step1();
		showStatus("step 2");
		step2();
		showStatus("step 3");
		step3();
		showStatus("step 4");
		step4();
	}

	private void showStatus(String message) {
		activate();
		Clients.showNotification(message);
		deactivate();
	}
</source>

== Cleanup after the operation ==

The callback '''onCleanup()''' is always called after the long operation has terminated (like a finally block) - here you can restore your UI (e.g. clearing a "busy" overlay).

<source lang="java" high="11, 16">

	@Command
	public void startLongOperation() {
		new LongOperation() {

			@Override
			protected void execute() throws InterruptedException {
				Thread.sleep(5000);
			}

			@Override
			protected void onCleanup() {
				Clients.clearBusy();
			};
		}.start();

		Clients.showBusy("Please wait, this may take some time.");
	}
</source>


== Cancel the operation ==

Use the '''cancel()''' method to notify your operation that it should interrupt and implement the '''onCancel()''' callback..

<source lang="java" high="12, 24, 34">
	private LongOperation longOperation;

	@Command
	public void startLongOperation() {
		new LongOperation() {
			
			@Override
			protected void execute() throws InterruptedException {
				//performing a long loop
				for(long i = 0; i < 100000000L; i++) {
					if(i % 1000 == 0) { // check every 1000 steps
						checkCancelled(); //will throw an InteruptedException and exit when cancelled from outside
					}
				}
			}

			@Override
			protected void onFinish() {
				//give the user some feedback the task is done
				Clients.showNotification("done sleeping for 5 seconds");
			};

			@Override
			protected void onCancel() {
				Clients.showNotification("operation aborted...");
			};
		}.start();
		
		Clients.showNotification("starting, you'll be notified when done.");
	}

	@Command
	public void cancelOperation() {
		longOperation.cancel();
	}
</source>

Many blocking operations (e.g. Thread.sleep(), Object.wait()) will naturally throw InterruptedException, otherwise you can call '''checkCancelled()''' any time between your steps or during long running loops.

= More examples = 

This section illustrates some recurring usage patterns for long operations, the full source is available for [[#Download | download]].

== A simple example ==

This simple example shows a basic use case of the '''LongOperation''' class. 
The operation creates a simple result which is added to the '''resultModel''' when it finishes. During the 3 seconds the default busy overlay is displayed, asking the user to wait.

[https://github.com/cor3000/zk-long-operations/blob/master/src/main/java/zk/example/longoperations/example/SimpleLongOperationViewModel.java SimpleLongOperationViewModel]
<source lang="java" high="10, 15, 26">
public class SimpleLongOperationViewModel {
	private ListModelList<String> resultModel = new ListModelList<String>();

	@Command
	public void startLongOperation() {
		LongOperation longOperation = new LongOperation() {
			private List<String> result;

			@Override
			protected void execute() throws InterruptedException {
				Thread.sleep(3000); //simulate a long backend operation
				result = Arrays.asList("aaa", "bbb", "ccc");
			}

			protected void onFinish() {
				resultModel.addAll(result);
			};

			@Override
			protected void onCleanup() {
				Clients.clearBusy();
			}
		};

		Clients.showBusy("Result coming in 3 seconds, please wait!");
		longOperation.start();
	}

	public ListModelList<String> getResultModel() {
		return resultModel;
	}
}
</source>

* '''Line 10:''' Implement the '''execute''' method to collect the result asynchronously
* '''Line 15:''' Implement the '''onFinish''' callback to update the UI once the operation has finished '''successfully'''
* '''Line 26:''' Launch the operation 

The ''''startLongOperation''''-command handler shows the "busy"-overlay to visually block user input. 
In '''onCancel''' it is cleared, how ever the long operation terminates (successful or not).

The straight forward zul code here is using the '''SimpleLongOperationViewModel''' and posting the '''startLongOperation'''-command

[https://github.com/cor3000/zk-long-operations/blob/master/src/main/webapp/longop-simple.zul longop-simple.zul]
<source lang="xml" high="3, 4">
    <div apply="org.zkoss.bind.BindComposer" 
    	 viewModel="@id('vm') @init('zk.example.longoperations.example.SimpleLongOperationViewModel')">
        <button onClick="@command('startLongOperation')" label="start"/>
        <grid model="@load(vm.resultModel)" height="300px"/>
    </div>
</source>

== Updating the UI during the Operation  ==

To update the UI during a long operation the desktop needs to be activated for UI updates. For this the methods '''activate()''' and '''deactivate()''' can be used as in the example below. It is advisable to activate the UI as short as possible for the UI to remain responsive. Any kind of UI updates can be performed between those 2 methods e.g. 
* change the busy message
* adding/removing in a ListModelList
* set any component properties (or notify change VM properties)
** e.g. update the value of a status-<label> or <progressmeter>
* show/hide/enable/disable UI elements dynamically
* post a global command MVVM (or post via EventQueue MVC)
* ...

[https://github.com/cor3000/zk-long-operations/blob/master/src/main/java/zk/example/longoperations/example/UpdatingStatusLongOperationViewModel.java UpdatingStatusLongOperationViewModel]
<source lang="java" high="21, 23, 38">
	private static final String IDLE = "idle";
	private String status = IDLE;

...

	@Command
	public void startLongOperation() {
		LongOperation longOperation = new LongOperation() {
			private List<String> result;

			@Override
			protected void execute() throws InterruptedException {
				step("Validating Parameters...", 10, 500);
				step("Fetching Data ...", 40, 1500);
				step("Filtering Data...", 60, 1750);
				step("Updating Model...", 90, 750);
				result = Arrays.asList("aaa", "bbb", "ccc");
			}

			private void step(String message, int progress, int duration) throws InterruptedException {
				activate();
				updateStatus(progress+ "% - " + message);
				deactivate();
				Thread.sleep(duration); //simulate processing time for the current step
			}

			@Override
			protected void onFinish() {
				resultModel.addAll(result);
				updateStatus(IDLE);
			}
		};
		longOperation.start();
	}

	private void updateStatus(String update) {
		status = update;
		BindUtils.postNotifyChange(null, null, UpdatingStatusLongOperationViewModel.this, "status");
	}

	public void getStatus() {
		return status;
	}
</source>

* '''Line 21:''' activate the thread for UI updates
* '''Line 23:''' deactivate the thread to send the updates back to the browser
* '''Line 38:'''  notify the change to update the UI

[https://github.com/cor3000/zk-long-operations/blob/master/src/main/webapp/longop-updating-status.zul longop-updating-status.zul]
<source lang="xml" high="5">

    <div apply="org.zkoss.bind.BindComposer"
    	 viewModel="@id('vm') @init('zk.example.longoperations.example.UpdatingStatusLongOperationViewModel')">
    	<button onClick="@command('startLongOperation')" label="start" 
    	        disabled="@load(vm.status ne 'idle')" autodisable="self" />
    	<label value="@load(vm.status)" />
    	<grid model="@load(vm.resultModel)" height="300px" />
    </div>
</source>

* '''Line 3:''' The status label to update during the operation

== Aborting a Long Operation ==

Referring to [https://github.com/cor3000/zk-long-operations/blob/master/src/main/java/zk/example/longoperations/example/CancellableLongOperationViewModel.java CancellableLongOperationViewModel] / [https://github.com/cor3000/zk-long-operations/blob/master/src/main/webapp/longop-cancellable.zul longop-cancellable.zul] you can cancel a long operation and give user feedback accordingly. The Long Operation will naturally terminate (calling onCancel) when the thread was interrupted, or by checking explicitly for cancellation between steps, in case the Long Operation was cancelled during a blocking method call.

Of course this requires you to break down your task into separate steps, or use [http://docs.oracle.com/javase/7/docs/api/java/lang/class-use/InterruptedException.html blocking method calls] that [http://www.ibm.com/developerworks/library/j-jtp05236/ throw InterruptedExceptions] themselves - the LongOperation class will handle them for you.

'''IMPORTANT''' [http://michaelscharf.blogspot.tw/2006/09/dont-swallow-interruptedexception-call.html don't swallow InterruptedExceptions] (they are very helpful)

<source lang="java" high="20, 21, 24, 30, 43">

	private LongOperation currentOperation;

	@Command
	public void startLongOperation() {
		currentOperation = new LongOperation() {

			@Override
			protected void execute() throws InterruptedException {
				step("Starting query (WHAT is the 'ANSWER';). This might take a about 7.5 million Years ...", 2000);
				step("Executing your query (1 million years passed) please wait...", 500);
				step("Executing your query (2 million years passed) please wait...", 500);
				step("Executing your query (3 million years passed) please wait...", 500);
...
				result = "The answer is 42";
			}

			private void step(String message, int duration) throws InterruptedException {
				//check explicitly if the task was cancelled 
				// if cancelled it will throw an InterruptedException to stop the task
				checkCancelled(); 
				activate(); //would throw an InterruptedException if cancelled
				updateStatus(message);
				deactivate();
				Thread.sleep(duration); //will throw an InterruptedException if cancelled during sleep
			}

...
			@Override
			protected void onCancel() {
				Clients.showNotification("Now you'll never know... be more patient next time");
			}

			@Override
			protected void onFinish() {
				Clients.showNotification(result);
			}
...
		}
	}

	@Command
	public void cancelOperation() {
		currentOperation.cancel();
	}
</source>

* '''Line 20:''' before performing a longer step call checkCancel()
* '''Line 21:''' activation will interrupt automatically if the operation was cancelled
* '''Line 24:''' many blocking method calls (such as sleep(), wait()) will interrupt naturally
* '''Line 30:''' UI callback for a cancelled Long Operation
* '''Line 43:''' cancel a task from a UI Command (MVVM or Event MVC)

== Parallel Tasks ==

The LongOperation class also supports parallel tasks which can be visualized separately e.g. using a <grid> with a ListModelList<TaskInfo> objects

check out the example: 
* [https://github.com/cor3000/zk-long-operations/blob/master/src/main/java/zk/example/longoperations/example/ParallelLongOperationViewModel.java ParallelLongOperationViewModel ]
* [https://github.com/cor3000/zk-long-operations/blob/master/src/main/webapp/longop-parallel.zul longop-parallel.zul]

[[File:parallel-longoperations.png]]

== Extending LongOperation ==
=== Decorate your Long Operation classes ===

It is easy to extend the LongOperation class to provide additional reusable functions for more compact code in your Composer/ViewModel classes, or to separate complex long operation code from your UI code.

[https://github.com/cor3000/zk-long-operations/blob/master/src/main/java/zk/example/longoperations/BusyLongOperation.java BusyLongOperation] shows a simpler way to update the busy message, and provides automatic cleanup once the task is done.
This results in a simpler usage and a centralized point in your application to determine how a "busy" message is displayed in the UI ([https://github.com/cor3000/zk-long-operations/blob/master/src/main/java/zk/example/longoperations/example/BusyLongOperationViewModel.java BusyLongOperationViewModel]).

=== Separate Long Operation code from UI code ===

Another reason to extend your own LongOperation class could be to reuse the same execute code in different ViewModels and only define how the UI reacts to the actual result:

see:
* [https://github.com/cor3000/zk-long-operations/blob/master/src/main/java/zk/example/longoperations/ResultLongOperation.java ResultLongOperation<RESULT>] - abstract basis class for LongOperations that have a result
* [https://github.com/cor3000/zk-long-operations/blob/master/src/main/java/zk/example/longoperations/example/DataFilterLongOperation.java DataFilterLongOperation] - specific implementation of ResultLongOperation
* [https://github.com/cor3000/zk-long-operations/blob/master/src/main/java/zk/example/longoperations/example/DataFilterLongOperationViewModel.java DataFilterLongOperationViewModel] - uses DataFilterLongOperation and updates the ViewModel when finished
* [https://github.com/cor3000/zk-long-operations/blob/master/src/main/webapp/longop-separate.zul longop-separate.zul]

= Summary =
The LongOperation class is a reusable/extensible base class for your long operations. It integrates with MVC/MVVM and helps to separate UI updates from Background processing still allowing intermediate UI updates.

'''What's next?'''

Of course this sample could be improved in several ways, using separate interfaces for the UI and the Task part or separate the LongOperation class into a Runner/Scheduler and LongOperationTask classes.

Functional programming in Java 8 could greatly improve the API.

Also the activate()/deactivate() blocks could slow down the long operation while waiting for the desktop to become available. (Which can be handled using  a separate UI Thread or queue for updates).

= Download =
* The source code for this article can be found in [https://github.com/cor3000/zk-long-operations github].

== Running the Example ==
The example consists of a maven web application project. It can be launched with the following command:

    mvn jetty:run

Then access the overview page http://localhost:8080/longoperations/overview.zul

And that's what you'll see:
<gflash width="900" height="600">long-operations-demo.swf</gflash>
[https://drive.google.com/a/potix.com/file/d/0B8UWhWsE_u68TGJTd0J6ZmZvSjA/view?usp=sharing Video for non-flash users.]


{{Template:CommentedSmalltalk_Footer_new|
|name=Potix Corporation
}}

[[Category:Small Talk]]