» DaoGen Forums
» Performance with List
Performance with List
Posted by anonymous 06.07.2004 03:48
Just I was going through your DaoGen 2 days back when I was looking for a good example that uses DAO design patterns
It seems yours is a good work developing a useful tool.
But I have once concern regarding the select query. You are using a List object to return results. If suppose I need to select data from a table that has huge volume of records, say 10,000 records, will it be effective to keep all of them in a List object ?
Will it be fine if you return an array of DTO objects rather than a List ?
Your feedback is highly appreciated
Re: Performance with List
Posted by Luigi 07.07.2004 00:00
> anonymous 06.07.2004 03:48
> Hi Tuomo
> Just I was going through your DaoGen 2 days back when I was looking for a good example that uses DAO design patterns
> It seems yours is a good work developing a useful tool.
Thanks for positive feedback Kay, I am happy to hear you like DaoGen!
> But I have once concern regarding the select query. You are using a List object to return results. If suppose I need to select data from a table that has huge volume of records, say 10,000 records, will it be effective to keep all of them in a List object ?
> Will it be fine if you return an array of DTO objects rather than a List ?
> Your feedback is highly appreciated
Well, I have used the DaoGen code in many projects - small and large. In one project, I implemented a command line batch-run program for handling replication of records from Oracle database to another different storage system (LDAP). In this case the DaoGen generates a list of ~200 000 objects from database, and then iterates through them applying changes and stores them to another system. This program works quite well, but it does need some memory and also the loading of the initial List takes some time.
I would say that the performance cost of JDBC is so huge, that it makes little difference to try to opimize the Dao-code. With List you can make more beautiful implementation with only a small performance cost. Also, if you do not have at least a moderate CPU with minimum 256 memory for JavaJVM, I suggest that you implement different algorithm for handling the desired 10 000 records - loading all of them to the memory (in any form) can make your code nice and easy to implement, but it certainly is not the most optimal solution for systems with limited resources.
However, loading all records to memory has the advantage that the the List (with value objects inside) is completely independent of the original database (and database connection). You can even close the actual connection and maybe connect to whole another database and then insert or update your records there (or maybe store the whole List to some other external storage).
By the way, the main reason for DaoGen to be like it is, was just the endless problems with poor programmers implementing Java database programs that ended messing up with open ResultSets and PreparedStatements. I hoped that this way the programmer responsible for the upper layer can not mess up the database resources.
Finally, if you want to compare the performance of List and Array, I suggest you do it like this:
- Generate the code for your tables, say Customer and CustomerDao
- Create a class CustomerDaoExt which extends CustomerDao and then implement new version of listQuery and loadAll methods so that they return arrays instead of Lists. You can use the existing methods as templates so this will require only small amount of code.
- Now you have both ways implemented and you can actually compare them.
Hope this helped you. Please feel free to post the results here, if you will compare the performance.