PDA

View Full Version : purging hands & its effect on db speed/performance



rowhousepd
01-25-2011, 12:25 AM
I know how to optimize by database, I've read the FAQ's on performance tweaks (http://faq.holdemmanager.com/questions/373/Holdem+Manager+Performance+%26+Speed+Tweaks), I get what I could do to my computer and to postgres to help HEM run quicker, but I'm not quite sure how purging hands effects speed.

According to one of the FAQ's (http://faq.holdemmanager.com/questions/196/Observed+Hand+History+Configuration), the "Purge Hand History Option here removes the Hand History, but keeps the stat." So I'm wondering then if it's going to help 1) make the HUD run more smoothly, and 2) make report queries run faster. I understand that the hand histories themselves will be gone, but if the stats are still there, is the db going to work better?

Here's an example of my situation: I had 4M hands in my db, and I purged all my old hands from last year which included some observed hands as well, then I optimized the db via HEM's maintenance tool. Now my db has a mere 50k hand histories (comparatively much smaller). Is it going to run at a peak performance now? Or are those other ~3.9 million hands somehow still clogging things up? If so, would exporting & reimporting the hands into a new db help things? Thanks.

Patvs
01-25-2011, 10:48 AM
it's going to help 1) make the HUD run more smoothly, and 2) make report queries run faster. I understand that the hand histories themselves will be gone




1/2: not really. It will only save you about 10-15% of diskspace.
Now in theory, if a huge database is smaller it will reduce access delays, but you'll only gain some milliseconds. When running a query this can show the biggest improvement... but a SLOW query running 15% faster will still be slow.

Try to manually vacuum/analyse/reindex the database in pgadminIII, and defrag the harddisk. Read:
http://forums.holdemmanager.com/manager-general/11194-top-10-hem-performance-increase-tips.html

rowhousepd
01-25-2011, 11:49 PM
If I did decide to recreate a new db and re-import hands (which I'd do by exporting hands histories from the old db, then importing them into the db), will the marked hands show up, or do exported hands only save the raw history?

Also, what about tournament summaries? When you export & re-import into a new db, do the summaries go with them? If not, is there another way to just save/import them? Thanks.

Patvs
01-26-2011, 09:50 AM
You'll lose the marked hands info.


You can export your tournament summaries separately, by selecting your tournies in the Tourney--> Results--> Dataview window--> export summaries.


On which site did you play all these games?
If PokerStars it's easier to just request ALL your hands / and summaries from Stars.

rowhousepd
01-26-2011, 03:34 PM
I play at FT. I'll give this a shot -- thanks for your help! :)

rowhousepd
01-26-2011, 05:51 PM
When you 'export the database' from the control panel it exports all of them in their original form and format. When you import them to a new database with the latest version of HM it should import them correctly.

You should import the original hands from your \hmarchive first and/or email your poker site support for all your original hands, before you import the exported versions, IMO.

When I have to rebuild my database I import the originals I have, the ones I have requested from support, and lastly my exported versions. By the time I import the exported hands it will show/skip many duplicate hands but this way I am sure I have all hands reimported.

Sorry if I'm being dense here, but let me see if I have this right. If I wished to create a pristine (and accurate) new DB w/ just the hands I wanted (not the ooooold hands from 2010 as I explained in my original post here) I would:



Purge the old hands I didn't want from my old DB.
Perform a vacuum/analyze.
Export all hands histories, & export tourney summaries.
Create the new DB.
Import the previously exported hands & tourney summaries.
Import original hh's that I want from the HEM archive and/or from the poker site once I request them.
Perform a vacuum/analyze.

Is that right? Will that ensure a clean (& hopefully smoother running) new DB?

Patvs
01-27-2011, 06:11 AM
Yes.... though I'm not a big fan of reimporting "exported hands".
(since some characteristics can get lots, for example if you export a RUSH hand, HoldemManager will no longer be able to recognize that hand as being a RUSH hand,... and will think it's a regular cash game.)

So importing original handhistories from your archive folder. (you can also request ALL your handhistories from Full Tilt) is best.

rowhousepd
01-27-2011, 08:11 PM
Good to know. Is this something you guys are going to fix in a future update? Also, are there other info you know about specifically that gets lost in an export?

Patvs
01-27-2011, 09:09 PM
Lost in export:
-RUSH
-maybe Multi Entry Tournaments will get lost

When exported: HoldemManager just moves all the handhistories to a .txt file.
The issue is with some hands it's not the content of the handhistory that's very interesting, it's the name of the actual file.

When exporting hands, HoldemManager currently is unable to also restore that name. (since it doesn't save that at all.... PokerTracker3 has the same issue)

---

In fact, the same thing happened when requesting RUSH hands from Full Tilt. (with their hand history beta) I'm unsure if Full Tilt fixed this on their side, since the requested RUSH hands look identical to non-RUSH hands.

So this isn't going to be fixed. And you should always save your original handhistories / backup your archive folder.

rowhousepd
02-02-2011, 05:14 AM
In fact, the same thing happened when requesting RUSH hands from Full Tilt. (with their hand history beta) I'm unsure if Full Tilt fixed this on their side, since the requested RUSH hands look identical to non-RUSH hands.

FYI, I created a new db and imported just the hands that was sent to me from Full Tilt when I requested them via their HH beta, and it looks like it's differentiating between Rush & non-rush hands now.

One other semi-related question: If I understand you right, the stats that players accumulate when you import hands stay with them (& I assume will be reflected in their HUD stats) even if you purge the actual hands that contributed to their stats. Is that correct?

Also, is the same true w/ observed hands? There are some players I have observed hands on, so if I purge all of my observed hands the stats that appear in their HUD are going to reflect all the hands I once observed but deleted since then, correct? Thanks.

Patvs
02-03-2011, 07:51 PM
If you use options--> settings--> observed HH configuration--> purge hands--> you can delete ALL the hands you want (observed, requested, etc.) and it will NOT interfere with the HUD stats. (all those deleted hands will still be used for the HUD stats)

rowhousepd
02-04-2011, 05:00 PM
If you use options--> settings--> observed HH configuration--> purge hands--> you can delete ALL the hands you want (observed, requested, etc.) and it will NOT interfere with the HUD stats. (all those deleted hands will still be used for the HUD stats)

Suppose you have a large db, and you purge some old hands that were both played by you and observed; you're left w/ and smaller db but one that has all the players' stats stored in the cache which are reflected in the HUD (even though you can't access the actual hand histories).

Now let's say you have a folder w/ HH's that have both new and duplicate hands. You import them all and have the hand histories which you can now review (again). But the question is will the villain's stats in the HUD accurate? That is, will the overall stats reflect an accumulation of the originally hands they played (before the purge) and some of the sames hands you imported again?

I ask because I know HEM won't import duplicate hands if the histories are already in the db, but I'm not sure if a player's stats in the cache will also filter out duplicates. Hope that makes sense.

Patvs
02-04-2011, 10:03 PM
It won't create duplicate hands used in the HUD stats.

Either: it will "restore" the handhistory, enabling you to replay the hand again, or it will just say "duplicate hand". (I'm unsure which it is) In either scenario it won't affect the stats.