Small Talks/2022/November/How To Load and Sort On Demand using Custom AbstractListModel and Sortable

Author
Edilson Alexandre Cuamba, Software Engineer, EXI - Engenharia e Comercialização de Sistemas Informáticos

Date
November 30, 2022

Version
ZK 9.6.1-Eval

Overview

Sometimes when you want to display data with a table structure, you use Listbox. This component is appropriate to it and works fine until your data gets huge and passes millions of rows; you will notice a little delay when you open the view with this component. For that reason, I needed to create a custom AbstractListModel that supports fetching only the data for the current page on the screen. In this article, I will show you how to build your own CustomAbstractListModel<T> that supports pagination at the database layer.

Let’s see what we will build:

External link to a GIF showing the application >>

Let’s do it!

Dependencies

To build this, we will need some functions. First is dependency injection (you can use other frameworks, but I personally like Spring) provided by Spring Framework. Second is the capacity to preserve the query structure and change the structure during the pagination. For this, I use a library called search-jpa-hibernate. Relax, and I will show you how to configure this!

This is my build.gradle file: https://github.com/EACUAMBA/loading_and_sorting_on__demand/blob/main/build.gradle

buildscript {
	repositories {
		mavenCentral()
	}
	dependencies {
		classpath("org.springframework.boot:spring-boot-gradle-plugin:2.5.12")
	}
}

apply plugin: 'java'
apply plugin: 'war'
apply plugin: 'org.springframework.boot'
apply plugin: "idea"

idea{
	module {
		setDownloadSources(true)
		setDownloadJavadoc(true)
	}
}

repositories {
	mavenLocal()
	maven { url "https://mavensync.zkoss.org/maven2" }
	maven { url "https://mavensync.zkoss.org/eval" }
	mavenCentral()
}

sourceCompatibility = '1.8'
targetCompatibility = '1.8'

ext {
	zkspringbootVersion = '2.5.12'
	springbootVersion = '2.5.12'
	zkspring = '4.0.0'
	zkVersion = '9.6.0'
	zatsVersion = '3.0.0'
	junitVersion = '4.13.1'
	searchJPAHibernateVersion = '1.2.0'
	lombokVersion = '1.18.24'
	mysqlVersion = '8.0.31'
	javaxServerletAPIVersion='4.0.1'
	commonsLang3Version='3.12.0'
}

configurations.testImplementation {
	// conflicts with ZATS (which is using jetty)
	exclude module: "spring-boot-starter-tomcat"
}

dependencies {
	implementation ("org.zkoss.zkspringboot:zkspringboot-starter:${zkspringbootVersion}")
	providedRuntime("javax.servlet:javax.servlet-api:${javaxServerletAPIVersion}")

	implementation ("org.zkoss.zk:zkspring-core:${zkspring}")
	implementation("org.zkoss.zk:zkplus:${zkVersion}")
	implementation ("org.springframework.boot:spring-boot-starter-data-jpa:${springbootVersion}")
	compileOnly ("org.springframework.boot:spring-boot-devtools:${springbootVersion}")
	implementation ("com.googlecode.genericdao:search-jpa-hibernate:${searchJPAHibernateVersion}")
	implementation("mysql:mysql-connector-java:${mysqlVersion}")

	compileOnly("org.projectlombok:lombok:${lombokVersion}")
	annotationProcessor("org.projectlombok:lombok:${lombokVersion}")
	testImplementation "org.springframework.boot:spring-boot-starter-test:${springbootVersion}"
	implementation("org.apache.commons:commons-lang3:${commonsLang3Version}")

	testImplementation "org.zkoss.zats:zats-mimic-ext96:${zatsVersion}"
	testImplementation "junit:junit:${junitVersion}"
}

These are some of the dependencies I like to use, so feel free to remove the ones you don't need. Now let’s build our Custom AbstractListModel<T>.

Building a PaginatedListModel<T>

First of all, I need to show you how a listbox works:

1. Every Listbox is associated with a ListModel if you don’t set one using Listbox.setModel(ListModel<T> listModel), your listbox will create one implicitly for your Listbox, so we will use this function to tell our Listbox which ListModel<T> it must use.
2. This ListModel<T> is an interface that has two important methods:

//Returns the value at the specified index.
public <E> getElementAt(i);

//Returns the length of the list.
public int getSize();

These two methods are the most important methods of our PaginatedListModel<T>; our class will override these two methods to call the specific data for a new row directly from the database, so we will reduce the use of a lot of resources and increase the velocity of our application. Let’s build this class.

Building the class PaginatedListModel<T>

Our class will be an implementation Sortable<T> and child of the class AbstractListModel<T> because we will need some methods that this class gives like getting the page size, getting the current page, and so on.

import com.eacuamba.loading_and_sorting.domain.service.Searchable;
import com.eacuamba.loading_and_sorting.helper.ApplicationContextHelper;
import com.googlecode.genericdao.search.Search;
import com.googlecode.genericdao.search.Sort;
import com.googlecode.genericdao.search.jpa.JPASearchFacade;
import org.apache.commons.lang3.ArrayUtils;
import org.zkoss.zk.ui.Desktop;
import org.zkoss.zk.ui.Executions;
import org.zkoss.zul.AbstractListModel;
import org.zkoss.zul.FieldComparator;
import org.zkoss.zul.event.ListDataEvent;
import org.zkoss.zul.ext.Sortable;

public class PaginatedListModel<T> extends AbstractListModel<T> implements Sortable<T> {
		//this three (respectively) are from the library that I use to search data and preserve the query during the life of the listbox.
    private final JPASearchFacade jpaSearchFacade;
    private final Search search;
    private final Sort[] sorts;

		//stores the key of our cache; you don't want to hit the database twice to get the same data, right?
    private final String CACHE_KEY = String.format("%s_cache", PaginatedListModel.class.getName());
    //stores the current object with the filters;
		private final T t;
		//start the size with null to know when to fetch or not.
    private Integer size = null;

    @SuppressWarnings({"unchecked"})
    public PaginatedListModel(T t, Class<T> tClass) {
				//tying to find an instance of JPASearchFacade, we need to create this bean.
        this.jpaSearchFacade = ApplicationContextHelper.getBean(JPASearchFacade.class);
				//tying to find our Searchable class with the logic or filter if we have it.
        this.search = ((Searchable<T>) ApplicationContextHelper.getBean(Searchable.class, t.getClass())).search(new Search(t.getClass()), t);
        //reset the sorts;
				this.sorts = null;
        //set our class.
        this.t = t;
        //reset the cache
        this.resetCache();
    }

    @SuppressWarnings({"unchecked"})
    public PaginatedListModel(T t, Sort... sorts) {
        this.jpaSearchFacade = ApplicationContextHelper.getBean(JPASearchFacade.class);
        this.search = ((Searchable<T>) ApplicationContextHelper.getBean(Searchable.class, t.getClass())).search(new Search(t.getClass()), t);
        this.sorts = sorts;
        this.t = t;
        this.resetCache();
    }

    public PaginatedListModel(T t, Boolean multiple, Sort... sorts) {
        this(t, sorts);
        //permits your select multiple rows.
        this.setMultiple(multiple);
    }

		// this method first try to get our cached Map<Integer, T>, if it finds it will try to get an element at a specific index, if not, it will hit the database, get the range of data, and then fill the cache and the cycle starts over again trying to find the data from the cache.  
    @Override
    @SuppressWarnings({"unchecked"})
    public T getElementAt(int index) {
				//getting the cache, it's a Map<Integer, T>, <index, our entity class>
        Map<Integer, T> cacheMap = getCache();
				
				//trying to get the object at the specified position;
        T target = cacheMap.get(index);
        if (Objects.isNull(target)) {
            List<T> page;

						//check if the user set sorts; if not, make a common search, then put in the page List<T>; otherwise do the same without the sort.
            //see the use of getActivePage() this will return the number of the current page, next, the use of getPageSize() it will return the size of page set at the .zul file.
						if (ArrayUtils.isEmpty(sorts)) {
                page = jpaSearchFacade.search(search.setPage(this.getActivePage()).setMaxResults(this.getPageSize()));
            } else
                page = jpaSearchFacade.search(search.setPage(this.getActivePage()).setMaxResults(this.getPageSize()).addSorts(sorts));

						//with the page insert the data in our cache, one by one, starting from the index the Listbox send to us.
            int nextIndex = index;
            for (T t : page) {
                cacheMap.put(nextIndex++, t);
            }
        } else {
            return target;
        }
				
				//returning the target if it was not in the cache;
        target = cacheMap.get(index);

				//check if the item was found. If not, throw an exception, sometimes it happens because of a change the number of data in the database, if so, you need to refresh the listbox!.
        if (Objects.isNull(target)) {
            throw new RuntimeException(String.format("The item at index %d was not found.", index));
        } else
            return target;
    }

    @Override
    public int getSize() {
				//to avoid hitting the database too many times to get the same value (the number of rows returned).
        if (Objects.isNull(size))
            return this.size = jpaSearchFacade.count(search);
        return size;
    }

		//this method returns the cache if it exists or creates one if not
    @SuppressWarnings({"unchecked"})
    private Map<Integer, T> getCache() {
        Desktop desktop = Executions.getCurrent().getDesktop();
        Map<Integer, T> cacheMap = (Map<Integer, T>) desktop.getAttribute(this.CACHE_KEY);
        if (Objects.isNull(cacheMap)) {
            cacheMap = new HashMap<>();
            desktop.setAttribute(this.CACHE_KEY, cacheMap);
        }
        return cacheMap;
    }

		//this will destroy the data in the cache, get the cache then clear the data if the cache exists;
    public void resetCache() {
				this.size = null;
        Desktop desktop = Executions.getCurrent().getDesktop();
        Map<Integer, T> cacheMap = (Map<Integer, T>) desktop.getAttribute(this.CACHE_KEY);
        if(Objects.nonNull(cacheMap))
            cacheMap.clear();
        fireEvent(ListDataEvent.CONTENTS_CHANGED, -1, -1);
    }

		//in this method the magic of sort in the database layer happen, it will implement the method sort from Sortable<T> interface and when the user click at the table header to sort it will receive the entity property to be used to sort and the direction, will save at our search object. 
    @Override
    public void sort(Comparator<T> tComparator, boolean ascending) {
        search.clearSorts();
        String rawOrderBy = ((FieldComparator) tComparator).getRawOrderBy();

        Arrays.stream(rawOrderBy.split(",")).map(String::trim).forEach(s -> {
            search.addSort(s, !ascending, true);
        });

        this.resetCache();
        this.clearSelection();
        fireEvent(ListDataEvent.CONTENTS_CHANGED, -1, -1);
    }


    @Override
    public String getSortDirection(Comparator<T> cmpr) {
        return null;
    }
}

Let’s build our ApplicationContextHelper

To help out find beans.

You can see that I used a class called ApplicationContextHelper; this class is my helper to find beans inside the Spring context. Let me show how this class is:

import org.springframework.beans.BeansException;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
import org.springframework.core.ResolvableType;
import org.springframework.stereotype.Component;
import org.yaml.snakeyaml.util.ArrayUtils;

//annotate it as a component, so spring can inject ApplicationContext in it using set method;
@Component
public class ApplicationContextHelper implements ApplicationContextAware {
    private static ApplicationContext applicationContext;
    @Override
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
        ApplicationContextHelper.applicationContext = applicationContext;
    }

		// this method receives a class and a type then find a bean of this class. With this type, we will be searching for something like CountrySearchable<Country>, country searchable implements a searchable interface.
    public static <T> T getBean(Class<T> tClass, Class<?> type){
        System.out.printf("Trying to find a bean of class %s and type %s.%n", tClass.getName(), type.getName());
        String[] applicationContextBeanNamesForType = ApplicationContextHelper.applicationContext.getBeanNamesForType(ResolvableType.forClassWithGenerics(tClass, type));
        return (T)applicationContext.getBean(applicationContextBeanNamesForType[0], tClass);
    }

    public static <T> T getBean(Class<T> tClass){
        System.out.printf("Trying to find a bean named %s.%n", tClass.getName());
        return applicationContext.getBean(tClass);
    }
}

Let’s build our bean JPASearchFacade

Now we need to create a bean called JPASearchFacade, so we can find this class in every place, to use it to find beans with some signatures:

import com.googlecode.genericdao.search.jpa.JPAAnnotationMetadataUtil;
import com.googlecode.genericdao.search.jpa.JPASearchFacade;
import com.googlecode.genericdao.search.jpa.JPASearchProcessor;
import org.springframework.context.annotation.Bean;
import org.springframework.stereotype.Component;

import javax.persistence.EntityManager;
import javax.persistence.PersistenceContext;

@Component
public class JPAFacadeConfiguration {
    @PersistenceContext
    private EntityManager entityManager;

    @Bean
    public JPASearchFacade getJpaSearchFacade(){
        final JPASearchFacade jpaSearchFacade = new JPASearchFacade();
        final JPAAnnotationMetadataUtil jpaAnnotationMetadataUtil = new JPAAnnotationMetadataUtil();
        final JPASearchProcessor jpaSearchProcessor = new JPASearchProcessor(jpaAnnotationMetadataUtil);
        jpaSearchFacade.setSearchProcessor(jpaSearchProcessor);
        jpaSearchFacade.setEntityManager(entityManager);
        return jpaSearchFacade;
    }
}

Searchable<T> where filter logic dwell

Now that you have these classes let’s build our view and use this new type of ListModel:

import com.googlecode.genericdao.search.Search;

public interface Searchable<T> {
		// we need the search so we can apply filters to it.
    Search search(Search search, T t);
}

First, you gonna need the Implementation of Searchable. Let’s build it:

@Service
public class CitySearchable implements Searchable<City> {

    @Override
    public Search search(Search search, City city) {
				//setting the type of result, set to auto, then clear old filters, add fetch to country so that it will execute only one query.
        search.setResultMode(ISearch.RESULT_AUTO);
        search.clearFilters();
        search.addFetch("country");

				//applying filters, 1st see if the name is not empty or null; if not, apply like filter with the right property.
        if(StringUtils.isNotBlank(city.getName())){
            search.addFilter(Filter.ilike("name", city.getName()));
        }

        return search;
    }
}

Our .zul file

<listbox id="listbox_city" mold="paging" pageSize="6">
			<listhead>
				<listheader label="Identifier" hflex="min"/>
				<listheader label="Name" sort="auto(name)"/>
				<listheader label="State Code" hflex="min"/>
				<listheader label="Country Name" sort="auto(country.name)"/>
				<listheader label="Phone Code" hflex="min"/>
				<listheader label="Currency Name"/>
				<listheader label="Region (Continent)" hflex="min"/>
			</listhead>

			<template name="model">
				<listitem value="${each}">
					<listcell label="${each.id}"/>
					<listcell label="${each.name}"/>
					<listcell label="${each.stateCode}"/>
					<listcell label="${each.country.name}"/>
					<listcell label="${each.country.phoneCode}"/>
					<listcell label="${each.country.currencyName}"/>
					<listcell label="${each.country.region}"/>
				</listitem>
			</template>
		</listbox>

Our controller

This controller will hold the logic behind the view.

@VariableResolver(org.zkoss.zkplus.spring.DelegatingVariableResolver.class)
public class CityPaginatedController extends SelectorComposer<Vlayout> {

    @WireVariable
    private CityRepository cityRepository;

    @Wire
    private Listbox listbox_city;

    @Override
    public ComponentInfo doBeforeCompose(Page page, Component parent, ComponentInfo compInfo) {
        return super.doBeforeCompose(page, parent, compInfo);
    }

    @Override
    public void doAfterCompose(Vlayout comp) throws Exception {
        super.doAfterCompose(comp);

//create an instance of PaginatedListModel of city
        final PaginatedListModel<City> cityPaginatedListModel = new PaginatedListModel<>(City.builder().build());

				//associeting with our listModel.
        listbox_city.setModel(cityPaginatedListModel);
    }
}

Download

If you want to check the running code, please access the repository at github.com. https://github.com/EACUAMBA/loading_and_sorting_on__demand

Summary

In this article, you saw that it is easy to create a custom ListModel that will handle queries and sort at database layer. As you can see, you have an AbstractListModel, and you still have the capabilities to select, get selection, and perform other things ALM<T> can do.

Please give it a try.

Note from ZK

Thanks to community user Edilson for writing this smalltalk and sharing his experience with everyone. If you also wish to contribute your work, don't hesitate to get in touch with us at info@zkoss.org.

Comments

Copyright © {{{name}}}. This article is licensed under GNU Free Documentation License.

</includeonly>