PDA

View Full Version : HEM v PT3 Imprt Speed Test



SwizzleStack
01-30-2009, 09:17 PM
I wanted to test import speed of both programs to see what I might be looking at if I wanted to import a large quantity of old hand histories. I realize that both programs are structured differently and have different recommended maintenance procedures which might need to be taken into account, as well as some other factors possibly. I also realize that when you are playing, how fast each program imports 100,000 hands isn't really relevant, but if you have a large quantity of old hands to import, or you play on multiple machines with multiple databases, etc, this test could be relevant to you.

As far procedure, I used the latest releases of both, PT3 V3.00 Build 4, and HEM v1.08 beta 6. I created new databases for both, and then HH text files (from Full Tilt) 100 megabytes at a time were imported, which was about 1200ish files per sample, and roughly 100,000 hands. In the first sample there were 99 duplicate hands. No duplicates in any of the other 4 samples. Also, fyi, each was done with no other programs running after a restart so as to compare processes similarly.

# Size Files Hands HEMtime PT3time
1 100 1220 101337 0:27:36 0:11:40
2 100 1235 102336 0:36:29 0:15:03
3 100 1227 101010 52:24.0 0:16:32
4 100 1212 101876 1:05:58 0:23:37
5 100 1250 102840 1:20:14 0:23:51

I also should note that in every sample except for the first one, HEM had less error hands than PT3. The error counts were 20,3,1,2,1 for HEM and 18,7,5,7,8 for PT3.

Lastly, After the second sample, I did the recommended db maintenance (vacuum & analyze HEM with PGadmin, and cluster, vacuum, analyze & cache PT3 in it's DB Housekeeping). It took 0:08:40 for HEM, and 0:10:14 for PT3 (0:03:19 for the vacuum & analyze).

I will do a few more samples for anyone that cares, but so far anyway, PT3 took less than 1/2 the time to import per sample, and HEM seems to be slowing down more as the db gets larger. Anyone who has any ideas how to speed up either, I'd love to hear ideas.

Thanks,

SwizzleStack

(I'll be posting this on both the PT3 & HEM forum, as well as possibly http://www.pokersoftware.com)

Rvg72
02-01-2009, 01:54 AM
A few things to keep in mind

1) Holdem Manager is processing a lot more information and updating more indexes and precompiling data for the HUD which is why everything runs so much faster in HM post import

2) We are doing all-in EV analysis on hands which makes a substantial difference in import speeds

3) in PT3 you are forced to do a lengthy cluster sequence which is not required in HM and even after the cluster sequence you should find that HM runs significantly faster running reports and pulling up data in the hud.

That all being said, we will be working on improving this in future versions and have some techniques we have been testing out that should make substantial differences.

Roy

SwizzleStack
02-01-2009, 03:56 AM
That all makes total sense to me. I just wanted to post some unbiased results of what I had done, and as I said in my post, I realize that once you have your db up and running, import speed won't be an issue, HUD and analysis processing speed will be much more important. But as I have millions of hands in multiple dbs, it is relevant to me currently.

Also, I would note that although it was only over a couple hundred thousand hands (and actually just did again for like 600,000 hands), the "lengthy cluster" process in PT3 really doens't add that much time. Doing cluster, vacuum, analyze, and updating the cache for PT3 versus using PGadmin to just vacuum and analyze HEM, the total times were pretty similar. We'll see how much PT3s time increases relative to HEM housekeeping as the db gets much larger.