Hashtable or Datatable?
in [VB.NET]
|
Prev: Help with Data Bound Controls
Next: Structure ?
From: Andrius B. on 26 Oct 2009 11:29 Hi all. I used Hashtable to store data loaded from Access (the db can contain from 1000 till 100000 records), and that worked very well, because of fast data retrieving from Hashtable using key. The main problem with Hashtable is sorting. I have to convert all the Hashtable to array list or smth, sort it with the help of ICompairer interface, and then converting back to hashtable. That takes much time. Now I am trying to modify the app code and started to use Datatable objects instead of Hashtable. Datatable has a much faster sorting mechanism, and that is very important for me, because I have to display the items from the db in Listview - not all of them, just some (up to 50), but the items must be ordered by certain key, t.i. the user should have possibility ti order the items in the way he wishes. That's why fast sorting is neccessary. ListView is in Virtual mode. But on the other hand, datatable is not the one I am looking for. The item retrieving from datatable (using datatable.rows.find (by key) is much slower than getting items from hashtable. When for sorting and displaying, it is not very bad, because, as I have said, I need at a moment dislpay only up to 50 items in Listview. But also my program has to make some calculations and analysis with the data, so when it tries to retrieve data from datatable, the speed of analyse decreases very markebly, in compairing to hashtable. So, what could I do, if want to have fast sorting and also fast data retrieving in my app? Storing the data both in datatable and in hashtable could be a solution, but it consumes to much memory. Is there a class or smth, that could provide this? Or should I try to write my own class implementing many interfaces (for sorting etc.)? I use VB.Net 2005. Thanks for any help.
From: Mr. Arnold on 26 Oct 2009 12:22 Andrius B. wrote: > Hi all. > > I used Hashtable to store data loaded from Access (the db can contain from > 1000 till 100000 records), and that worked very well, because of fast data > retrieving from Hashtable using key. > The main problem with Hashtable is sorting. I have to convert all the > Hashtable to array list or smth, sort it with the help of ICompairer > interface, and then converting back to hashtable. That takes much time. > Now I am trying to modify the app code and started to use Datatable objects > instead of Hashtable. Datatable has a much faster sorting mechanism, and > that is very important for me, because I have to display the items from the > db in Listview - not all of them, just some (up to 50), but the items must > be ordered by certain key, t.i. the user should have possibility ti order > the items in the way he wishes. That's why fast sorting is neccessary. > ListView is in Virtual mode. > But on the other hand, datatable is not the one I am looking for. The item > retrieving from datatable (using datatable.rows.find (by key) is much slower > than getting items from hashtable. When for sorting and displaying, it is > not very bad, because, as I have said, I need at a moment dislpay only up to > 50 items in Listview. But also my program has to make some calculations and > analysis with the data, so when it tries to retrieve data from datatable, > the speed of analyse decreases very markebly, in compairing to hashtable. > Yes, a datatable is as slow as it gets and not very scalable. http://msdn.microsoft.com/en-us/library/dd364983.aspx <copied> At the end of the day, however, if you�re using DataTables, the Select method is one of the worst things you could possibly do to limit the scalability of your application. We�ve discussed several alternatives here, including LINQ, indexing tables to make use of Rows.Find, and morphing the DataTable into a generic collection. Any of these options will be orders of magnitude faster than Select for a set of records of reasonable size. You also want to consider paring your set of records down as much as possible to speed up both aggregation and selection of records. <end copy> > So, what could I do, if want to have fast sorting and also fast data > retrieving in my app? Storing the data both in datatable and in hashtable > could be a solution, but it consumes to much memory. Is there a class or > smth, that could provide this? Or should I try to write my own class > implementing many interfaces (for sorting etc.)? > I use VB.Net 2005. The example is in C# in the linq, but you can do the same in VB using List<T>, a generic collection, which can be done VB2005. http://dotnetslackers.com/Community/blogs/simoneb/archive/2007/06/20/How-to-sort-a-generic-List_3C00_T_3E00_.aspx It's get/set in VB as opposed to let/set in C#. http://www.java2s.com/Tutorial/VB/0120__Class-Module/PropertieswithGetterandSetter.htm
From: Scott M. on 26 Oct 2009 14:05 "Mr. Arnold" <Arnold(a)Arnold.com> wrote in message news:utnHehlVKHA.4704(a)TK2MSFTNGP06.phx.gbl... > It's get/set in VB as opposed to let/set in C#. C# uses get/set. -Scottt
From: Andrius B. on 26 Oct 2009 14:09 Thanks Mr. Arnold! I tried to realize Your idea to use only List (of type) to store the data and then to sort it using to List.Sort(IComparer). Sorting became a little bit faster then using Hashtable and the indirect sorting (by convert to arraylist adn back to Hashtable, as I explained in my question). Of course, the new conception coud not reach the speed of sorting produced by DataTable :) I thing I could find some changes in then inner code witch does the comparing (inside the Compare function) to make the sorting faster. Thanks again for the suggestion. Nevertheless, it would be better to have some kind of class that could combine the good things of DataTable and generic objects :) Regards. "Mr. Arnold" <Arnold(a)Arnold.com> wrote in message news:utnHehlVKHA.4704(a)TK2MSFTNGP06.phx.gbl... > Andrius B. wrote: >> Hi all. >> >> I used Hashtable to store data loaded from Access (the db can contain >> from >> 1000 till 100000 records), and that worked very well, because of fast >> data >> retrieving from Hashtable using key. >> The main problem with Hashtable is sorting. I have to convert all the >> Hashtable to array list or smth, sort it with the help of ICompairer >> interface, and then converting back to hashtable. That takes much time. >> Now I am trying to modify the app code and started to use Datatable >> objects >> instead of Hashtable. Datatable has a much faster sorting mechanism, and >> that is very important for me, because I have to display the items from >> the >> db in Listview - not all of them, just some (up to 50), but the items >> must >> be ordered by certain key, t.i. the user should have possibility ti order >> the items in the way he wishes. That's why fast sorting is neccessary. >> ListView is in Virtual mode. >> But on the other hand, datatable is not the one I am looking for. The >> item >> retrieving from datatable (using datatable.rows.find (by key) is much >> slower >> than getting items from hashtable. When for sorting and displaying, it is >> not very bad, because, as I have said, I need at a moment dislpay only up >> to >> 50 items in Listview. But also my program has to make some calculations >> and >> analysis with the data, so when it tries to retrieve data from datatable, >> the speed of analyse decreases very markebly, in compairing to >> hashtable. >> > > Yes, a datatable is as slow as it gets and not very scalable. > > http://msdn.microsoft.com/en-us/library/dd364983.aspx > > <copied> > At the end of the day, however, if you�re using DataTables, the Select > method is one of the worst things you could possibly do to limit the > scalability of your application. We�ve discussed several alternatives > here, including LINQ, indexing tables to make use of Rows.Find, and > morphing the DataTable into a generic collection. Any of these options > will be orders of magnitude faster than Select for a set of records of > reasonable size. You also want to consider paring your set of records down > as much as possible to speed up both aggregation and selection of records. > > <end copy> > >> So, what could I do, if want to have fast sorting and also fast data >> retrieving in my app? Storing the data both in datatable and in hashtable >> could be a solution, but it consumes to much memory. Is there a class or >> smth, that could provide this? Or should I try to write my own class >> implementing many interfaces (for sorting etc.)? >> I use VB.Net 2005. > > The example is in C# in the linq, but you can do the same in VB using > List<T>, a generic collection, which can be done VB2005. > > http://dotnetslackers.com/Community/blogs/simoneb/archive/2007/06/20/How-to-sort-a-generic-List_3C00_T_3E00_.aspx > > It's get/set in VB as opposed to let/set in C#. > > http://www.java2s.com/Tutorial/VB/0120__Class-Module/PropertieswithGetterandSetter.htm
From: Göran Andersson on 26 Oct 2009 14:27
Scott M. wrote: > "Mr. Arnold" <Arnold(a)Arnold.com> wrote in message > news:utnHehlVKHA.4704(a)TK2MSFTNGP06.phx.gbl... > >> It's get/set in VB as opposed to let/set in C#. > > C# uses get/set. > > -Scottt > Besides, Let and Set is the same thing. VB6 uses Get and Let, but VB.NET uses Get and Set. -- G�ran Andersson _____ http://www.guffa.com |