Lazy Loading DataTable with IceFaces
08:38 PMDevelopment, JavaStefano
IceFaces provides numerous components to facilitate the development of web applications.
Among these, one of the most ‘useful’ is definitely ice:dataTable.
This component, together with ice:dataPaginator makes it possible to paginate the entire data set of a table, showing only N rows per page.
The main weakness of this component lies in the difficulty to handle large data sets: ice:dataTable component need to receive a list containing all rows that will gradually showed.
As long as we are in the order of some hundreds of records, there is no problem to provide the entire results list to the component; but if they begin to be thousands, keep in memory such a quantity of objects can be an issue.
The solution is to manage the list of results in a lazy way: the list itself will retrieve the records to show in the current page and only when there will be a real need.
Besides this, we need also to manage the total page number: the paginator in fact invokes the method size() of the list supplied to the table calculates the number of pages.
Let’s see three possible implementations to implement the lazy loading of the list.
They will be initialized in the page backingBean and used in this way in the .jsf template:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | <ice:dataTable id="data" value="#{backingBean.lazyLoadingList}" rows="10" > .... </ice:dataTable> <ice:dataPaginator id="scroll_1" for="data" fastStep="10" pageCountVar="pageCount" pageIndexVar="pageIndex" paginator="true" paginatorMaxPages="9"> ... </ice:dataPaginator> |
1. LazyLoadingList
This list keeps in memory a list of objects corresponding to the first page. When the user navigates in the following pages, they will be retrieved by method get() and saved in a different list. Note that the constructor of the list accepts as incoming parameter totalResultsNumber, that is the total number of results to show in the table. This parameter ‘is usually the result of a count query.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 | package com.devinterface.lazyloading; import java.util.AbstractList; import java.util.List; pre /** * This list loads and stores only the first page and the current page. * If pageSize is equals to totalResultsNumber, the dataTable will be non paginated: the first query will retrieve all the dataset. * * @param <T> */ public class LazyLoadingList<T> extends AbstractList<T> { private IDataProvider<T> dataProvider; private List<T> firtsPageData; private List<T> currentPageData; private int currentPage = -1; private int totalResultsNumber; private int pageSize; /** * @param dataProvider, the object that will perform the query * @param pageSize, the number of rows to be showed in a table page * @param totalResultsNumber, the total number of rows as result of the database count query. */ public LazyLoadingList(IDataProvider<T> dataProvider, int pageSize, int totalResultsNumber) { this.dataProvider = dataProvider; this.totalResultsNumber = totalResultsNumber; this.pageSize = pageSize; } @Override public T get(int i) { if (i < pageSize) { if (firtsPageData == null) firtsPageData = dataProvider.getBufferedData(i, pageSize); return firtsPageData.get(i); } int page = i / pageSize; if (page != currentPage) { currentPage = page; currentPageData = dataProvider.getBufferedData(i, pageSize); } return currentPageData.get(i % pageSize); } @Override public int size() { return totalResultsNumber; } public void setTotalResultsNumber(int totalResultsNumber) { this.totalResultsNumber = totalResultsNumber; } @Override public void clear() { firtsPageData.clear(); currentPageData.clear(); } } |
2. LazyLoadingMapList
This list keeps retrieved records in a HashMap. Note that the map is never cleared, so in the worst case (the user scrolls one by one all the pages) it will contains all elements of the dataset.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 | package com.devinterface.lazyloading; import java.util.AbstractList; import java.util.HashMap; import java.util.List; import java.util.Map; /** * This list stores all records in a map with key = "table row index" and value = "T" * If pageSize is equals to totalResultsNumber, the dataTable will be non paginated: the first query will retrieve all the dataset. * Note that if a user moves from first to last page, one page at time, all record will be in memory. * * @param <T> */ public class LazyLoadingMapList<T> extends AbstractList<T> { private IDataProvider<T> dataProvider; private int totalResultsNumber; private int pageSize; /** cache of loadedData items */ private Map<Integer, T> loadedData; /** * @param dataProvider, the object that will perform the query * @param pageSize, the number of rows to be considered as "a page" * @param totalResultsNumber, the total number of rows as result of the database count query. */ public LazyLoadingMapList(IDataProvider<T> dataProvider, int pageSize, int totalResultsNumber) { this.dataProvider = dataProvider; this.totalResultsNumber = totalResultsNumber; this.pageSize = pageSize; loadedData = new HashMap<Integer, T>(); } @Override public T get(int i) { if (!loadedData.containsKey(i)) { int pageIndex = i / pageSize; List<T> results = dataProvider.getBufferedData(i, pageSize); for (int j = 0; j < results.size(); j++) { loadedData.put(Integer.valueOf(pageIndex * pageSize + j), (T) results.get(j)); } } return loadedData.get(i); } @Override public int size() { return totalResultsNumber; } public void setTotalResultsNumber(int totalResultsNumber) { this.totalResultsNumber = totalResultsNumber; } @Override public void clear() { loadedData.clear(); } } |
3. LazyLoadingBufferedMapList
This list represents an evolution of the previous one, keeping in memory only the elements corresponding to the current page, the previous and the next.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 | package com.devinterface.lazyloading; import java.util.AbstractList; import java.util.HashMap; import java.util.List; import java.util.Map; /** * This list stores records in a map with key = "table row index" and value = "element" * It keeps a buffer of 3 pages: the first time it will load pages 1,2,3. * When user moves to page 4, all stored data will be cleared and will be retrieved page 3,4,5. * Basically, this list will store current page, previous page and next page. * If bufferSize is equals to totalResultsNumber, the dataTable will be non paginated: the first query will retrieve all the dataset. * * @param <T> */ public class LazyLoadingBufferedMapList<T> extends AbstractList<T> { private IDataProvider<T> dataAdapter; private int totalResultsNumber; private int pageSize = 10; private int bufferSize = 30; /** cache of loadedData items */ private Map<Integer, T> loadedData; /** * * @param dataAdapter, the object that will perform the query * @param pageSize, the number of rows to be considered as "a page" * @param totalResultsNumber, the total number of rows as result of the database query. */ public LazyLoadingBufferedMapList(IDataProvider<T> dataAdapter, int pageSize, int totalResultsNumber) { this.dataAdapter = dataAdapter; this.totalResultsNumber = totalResultsNumber; this.pageSize = pageSize; this.bufferSize = pageSize * 3; loadedData = new HashMap<Integer, T>(); } @Override public T get(int i) { if (!loadedData.containsKey(i)) { clearMap(); int startRow = getStartRow(i); int numElementToFind = bufferSize; if ((startRow + numElementToFind) > totalResultsNumber) numElementToFind = totalResultsNumber - startRow; List<T> results = dataAdapter.getBufferedData(startRow, numElementToFind); for (int j = 0; j < results.size(); j++) loadedData.put((startRow + j), (T) results.get(j)); } return loadedData.get(i); } /** * clears the map except the first element that MUST be kept */ private void clearMap() { T firstElement = loadedData.get(0); loadedData.clear(); loadedData.put(0, firstElement); } /** * Calculates the index of the previous page's first element * @param i, the current row index * @return the index of the previous page's first element */ private int getStartRow(int i) { int currentPage = (i / pageSize) + 1; int firstIndexOfCurrentPage = pageSize * (currentPage - 1); int firstIndexOfPreviusPage = firstIndexOfCurrentPage - (bufferSize / 3); if (firstIndexOfPreviusPage < 0) firstIndexOfPreviusPage = 0; return firstIndexOfPreviusPage; } @Override public int size() { return totalResultsNumber; } public void setNumResults(int numResults) { this.totalResultsNumber = numResults; } @Override public void clear() { loadedData.clear(); } } |
Conclusions
All implementations give the opportunity to retrieve records in a lazy way. Surely the second solution is the most potentially weak as in the worst case will keep in memory all the dataset.
Surely the third implementation is the best solution from all points of view. It allows to have in memory only a limited number of records and to meet any potential “back and forth” of the user.
Tags: dataTable, java, jsf, pagination
About DevInterface
We are an information and communication technology agency. Our mission is to provide web application development, design services and communication strategies. We specialize in building web applications with modern and efficient frameworks.
Random Posts
18 Responses to “Lazy Loading DataTable with IceFaces”
Leave a Reply
Insert code beetween <code lang="ruby"> and </code>
-
our LINKS
[...] Original Post [...]
works great! thanks!
i’m trying to extend this with sorting mechanism.. could you give me a hint?
Hi Martin.
Suppose to have a sortable column.
The best way to handle sorting is to add an actionListener to the ice:commandSortHeader that clears the list (e.g. calls the clear() method on the lazy list) and forces to load the page again.
did it =)
0}”
sortColumn=”#{universalRequestBean.sortColumnName}”
sortAscending=”#{universalRequestBean.ascending}”>
in the Getter of the lazy list i check changes of the properties ’sortColumnName’ and ‘ascending’. the list reloads data if necessary
oh, there happened something to the ice:datatable – Tag in my post ^^
the important properties here again:
value=”#{universalRequestBean.lazyFadsList}”
sortColumn=”#{universalRequestBean.sortColumnName}”
sortAscending=”#{universalRequestBean.ascending}”
Hi,
I thought I’d post my question here because I’ve got a problem related to datatable.
Before I turn to the paginator solution you excellently describe I’d like to find out whether I’m not doing something wrong.
The problem is I’m using a datatable to display around 7000 ISO language codes.
I’m not using pagination at the moment. It takes a while, as expected, to display the list. I can see that the majority of the work is located in the browser (IE or FF) because I see the processer shooting up to 99% for the IE/FF process.
I could live with waiting a bit longer to retrieve the list, because it’s big.
However, sorting is very very slow (5min) and I’m wondering what could be the cause ? Any idea’s why sorting is much more slower than displaying the list for the first time ? Are there any icefaces settings that could resolve this ?
thanks,
EDH
Hi EDH.
When you call a column sort the dataTable will invoke an alphanumeric sorting on the value of selected column.
This sorting should be handled on server side by calling the compareTo method of your field’s class.
Then it should display again the page.
So the total time should be given by:
- the time to perform all 7 phases of JSF lifecycle (maybe one of them will take too much time)
- sort the dataset (be sure to not perform the query a second time)
- render the table again
I suggest to implement the javax.faces.event.PhaseListener in order to log the total time spent on each phase.
I need help, I can send the full code. I look at BackingBean and all implementation.
thanks
Hello!
I have a problem with ICEFACES JSF implementation.
Even if dataProvider.getBufferedData return a page section of records(10) and set correctly totalResultsNumber to (200), the paginator still display count of 10 records and one page. It seems that it does not use size() function for total number, but use an internal count of records.
Please help with this issue!
Im trying to implement your paginator, but i dont undertand what to do with the IDataProvider->
its a custom class? with the query?
where do i put the query to the database, and how i put in the backing bean the lazyloading list???????
@Cabaji: the IDataProvider must be implemented by a your class in order to perform the paginated query.
You must implement the method
that gets as input a start row and the number of row to fetch and returns a list that contains offset elements
Thx works great!, incredible code, but can u explain to me how is the interaction of the paginator with the database?,
Hi i tried your lazy loading paginator…> but when i debugged the application, i noticed, that calls the database a lot of times, when press search to load the database, so im looking that its doing nothing when i click next in the datapaginator. Im doing something wrong?
thanks .
Hi, I’d like to use this code in my school project. Can I do this?
Krystian, of course, you can use this code in your project.
It could have a MIT License
Hi,
gr8..piece of code
help me a lot
Thanks.
Could you provide a full example code.. JSF pages, BackingBean..etc..
Thanks!