Single HM3 database to 2 ('Lifetime' and 'Current'). Renaming, duplicating, purging?
Results 1 to 7 of 7
  1. #1
    Junior Member
    Join Date
    Feb 2015
    Posts
    13

    Question Single HM3 database to 2 ('Lifetime' and 'Current'). Renaming, duplicating, purging?

    Hi,

    I'm unsure if what I'm wanting to do is possible in HM3 (yet) but if it is I'd be very grateful to learn how to do it.

    What I'm wanting to do is go from my current practice of keeping a single lifetime database to having two databases, one "Lifetime Database" (everything ever, for full analysis) and one "Current Database" (the most recent x months, for general use and playing, and to help HM3 performance in typical usage outside of full analysis - hopefully I'm correct in assuming that a database stripped of a chunk of hands yet retaining villain stats is both possible and would imply reduced memory usage).

    In my head my plan for doing this involves i) duplicating my single lifetime database; ii) purging hands (not stats, just hands) older than x months in the duplicated one I'll use as the "Current Database"; iii) renaming the databases; iv) periodically merging/importing the new hands of the "Current Database" into the "Lifetime Database." to keep it up to date.

    If I understand purging correctly, purging hand histories (or at least, doing so in a specific way) does not remove the accumulated villain stats for HUD purposes, so the new current database would still allow me to have the same HUD stats I've collected on opponents - I'd just not be able to analyse/replay the hands purged from the streamlined database, but would still be able to do that in the lifetime one. If my understanding isn't correct on this, it would change the viability of doing this in the first place, as I wouldn't want to lose so much HUD information. Similarly, since I'm assuming that keeping a lifetime and a current database in this way would result in better performance (faster loading of the current database vs lifetime database, less RAM usage within the current database vs lifetime database etc) if this assumption is incorrect, then the value to me of switching from a single lifetime to a lifetime plus a current database would be reduced.


    So my first questions are to verify the correctness of two assumptions that my plan is predicated upon:

    1) Does purging hand histories in HM3 (or using purge in a specific way) just purge the hands for analysis/replay purposes whilst ALL villain stats are retained for HUD purposes?

    2) Would doing the above result in HM3 performance gains whilst using the current database relative to the lifetime database (quicker loading of the smaller current database, lower ram usage, etc)?


    If the answers to 1 and 2 are affirmative (and therefore there's value with me making this change), then in terms of practically actioning my plan:

    3) What's the best (priorities being reliability > speed > ease) way to make a new duplicate of an existing database? (Exporting all hands and reimporting? Creating a new database and importing hands? Creating a new database and merging? - not sure if such an option exists - Selecting the file in Windows and copy pasting? - not sure if that would even work)

    4) What are the specific steps to ensure that when purging hands in the duplicated database (to make it a current one) I do so in a way that retains my villain stats for HUD purposes?

    5) How do I rename databases? I can't find an option within HM3 to do this, and I'm not sure if renaming the actual .hmdb file in Windows Explorer would do the trick or be a terrible idea.

    6) For the purposes of the regular periodic updating of the Lifetime database with the new hands from the Current database, I'm not finding a merge or sync database option in HM3. Would the best way to do it therefore be to use good ol fashioned import hands or something else?


    Thanks (and sorry for so many questions, I'm fresh to HM3 and wanting to get to grips with it ASAP so I can be fully comfortable and efficient using it and get back to improving my poker )


    PS. I know this is a general support forum and I am seeking general support, but the context of this does lend itself to a few feature suggestions for consideration (basically stuff that if it was in HM3 I would have clicked through and used to accomplish what I'm trying to do here), so I hope it's okay for me to append them here, and if appropriate I could cut/paste this bit into the feature forum: [File Menu > Rename Database] [File Menu > Duplicate Database - perhaps too niche use, but here I am wanting to do it!], [greater clarity for uses wanting to purge hands whilst retaining their valuable villain stats for their HUD - could be realized either within the software or by a guide/tutorial - assuming my understanding of purging is correct ofc], [an automated (set and forget) or semi-automated (a few clicks) way of keeping a lifetime and current database and having the new hands from current make their way into lifetime - in my head I see the "sync to cloud" feature of HM2 repurposed to a "sync to other database" feature in HM3, or a merge feature - perhaps too niche use, but here I am thinking it would be handy for me, especially if my plan above is viable].

  2. #2
    *** HM3! *** fozzy71's Avatar
    Join Date
    Jun 2005
    Location
    HM Support
    Posts
    32,803

    Default

    Quote Originally Posted by Demonic666 View Post
    Hi,

    I'm unsure if what I'm wanting to do is possible in HM3 (yet) but if it is I'd be very grateful to learn how to do it.

    What I'm wanting to do is go from my current practice of keeping a single lifetime database to having two databases, one "Lifetime Database" (everything ever, for full analysis) and one "Current Database" (the most recent x months, for general use and playing, and to help HM3 performance in typical usage outside of full analysis - hopefully I'm correct in assuming that a database stripped of a chunk of hands yet retaining villain stats is both possible and would imply reduced memory usage).

    In my head my plan for doing this involves i) duplicating my single lifetime database; ii) purging hands (not stats, just hands) older than x months in the duplicated one I'll use as the "Current Database"; iii) renaming the databases; iv) periodically merging/importing the new hands of the "Current Database" into the "Lifetime Database." to keep it up to date.

    If I understand purging correctly, purging hand histories (or at least, doing so in a specific way) does not remove the accumulated villain stats for HUD purposes, so the new current database would still allow me to have the same HUD stats I've collected on opponents - I'd just not be able to analyse/replay the hands purged from the streamlined database, but would still be able to do that in the lifetime one. If my understanding isn't correct on this, it would change the viability of doing this in the first place, as I wouldn't want to lose so much HUD information. Similarly, since I'm assuming that keeping a lifetime and a current database in this way would result in better performance (faster loading of the current database vs lifetime database, less RAM usage within the current database vs lifetime database etc) if this assumption is incorrect, then the value to me of switching from a single lifetime to a lifetime plus a current database would be reduced.
    This is not possible in HM2 or HM3. It might have been possible in PT4? or HM1?


    So my first questions are to verify the correctness of two assumptions that my plan is predicated upon:

    1) Does purging hand histories in HM3 (or using purge in a specific way) just purge the hands for analysis/replay purposes whilst ALL villain stats are retained for HUD purposes?

    2) Would doing the above result in HM3 performance gains whilst using the current database relative to the lifetime database (quicker loading of the smaller current database, lower ram usage, etc)?
    None of the above is possible. If you purge a hand all the data/stats for every player involved in that hand are purged from the database.


    If the answers to 1 and 2 are affirmative (and therefore there's value with me making this change), then in terms of practically actioning my plan:

    3) What's the best (priorities being reliability > speed > ease) way to make a new duplicate of an existing database? (Exporting all hands and reimporting? Creating a new database and importing hands? Creating a new database and merging? - not sure if such an option exists - Selecting the file in Windows and copy pasting? - not sure if that would even work)

    4) What are the specific steps to ensure that when purging hands in the duplicated database (to make it a current one) I do so in a way that retains my villain stats for HUD purposes?
    If you want a true copy of a database just close HM3, exit the HM3 server and copy/paste the [HM3Database].hmdb file. Or use the Tools - Backup/Restore function, or manual import of the same files if you have them easily accessible and well organized. Export hands from one DB and import to another is the last choice IMO, when none of the other faster and easier methods worked for some reason.


    5) How do I rename databases? I can't find an option within HM3 to do this, and I'm not sure if renaming the actual .hmdb file in Windows Explorer would do the trick or be a terrible idea.
    Close HM3/Server and use Windows File Explorer and rename the [HM3Database].hmdb file in your \Documents\Holdem Manager 3\Databases folder. If you rename the active database this will force the HM3 Setup Wizard the next time you launch HM3 and give you the options to open an existing DB or create a new one, etc.


    6) For the purposes of the regular periodic updating of the Lifetime database with the new hands from the Current database, I'm not finding a merge or sync database option in HM3. Would the best way to do it therefore be to use good ol fashioned import hands or something else?
    Import any hands from the \HM3Archive to the database that were played/archived since the last hand in the database, just like you would HM2. We do have a feature request for a cloud sync feature for HM3 like we had in HM2 (that didn't work well for everyone) but it would be a sync system, not a merge system.

    If you play poker on multiple computers you can manually keep two databases in sync with a shared archive folder of original hand history files to manually import from, but you can NOT* keep your active databases hmdb files on a cloud hosting service or they will be corrupted. Those that are very careful and organized could conceivably just use the same hmdb file (or copies of the file) and move it from machine to machine whenever they move to a new HM3 installation.

    The only way to sync them currently is manually. You would need to keep your \HM3Archive folder on both machines sync'd via dropbox (or similar cloud hosting service), and manually import recent hands to each computer like we suggest for HM2 -*http://hm2faq.holdemmanager.com/ques...+Using+Dropbox

    You can use the Tools - Backup/Restore Database menu option for the initial transfer of the database but after that you will have to manually import hands to keep them in sync.

    *Please do not create your HM3 Database in a sub-folder of any cloud file hosting service (Dropbox, OneDrive,.Google Drive, etc) as it can cause database corruptions because it will constantly be syncing the database to the cloud while you are playing/importing.* You can configure HM3 to archive any auto imported hand history files to a cloud file hosting service folder if you choose to as that will not affect the SQLite database in any way, and you can obviously store any HM3 database backup files you create in a cloud folder instead of the default path HM3 suggests when backing up databases.

  3. #3
    Junior Member
    Join Date
    Feb 2015
    Posts
    13

    Default

    Thanks for the thorough answer Fozzy. Even though I can't do exactly what I intended (at least not with the exact results I was wanting), I have learned how to do some useful things and have the necessary info to do a revised compromised version of it, so I'm grateful.

    I may do such a revised version of my plan, though I'll need to think about how many hands worth of villain stats I'm prepared to jettison to form my slimmed down current/play database. I suppose beyond the performance gains to be had, there are a few convincing arguments one could make for ditching their hands over x months/years old: some villains will have moved on from the game or to different stakes, the player themself might have gone up (or down) a stake level and be against a mix of new villains and old villains who may (should) play differently at that stake level, and villains will (unless they are totally negligent poker players - whom I'd love to meet more of!) have developed their play over time making the older data a less accurate reflection of their current play. Still, it feels like a tough strategic decision; the game screams out for players to harness every single bit of an edge they can, and it'll feel like shooting myself in the foot to not be using a large portion of data I've got in the full database, and certainly some of the stats I use become massively less reliable without a large sample size. But I do think maybe the hands I've got that are over a year old might have so much less value relative to more recent ones that it might be worth leaving them out of a current play database for the other benefits and considerations.

    I'll search that massive internet thing for giudance on it, but if anyone knows of any good informed articles on the relative worth of older vs newer hand history data, I'd be interested to read recommendations. Might make the decision easier and the choice of a purge cut-off smarter than I might come up with myself.

  4. #4
    Junior Member
    Join Date
    Feb 2015
    Posts
    13

    Question Purging appears to get stuck at 85%, same with two different size selections

    Hi,

    So I've made a copy of my big database and renamed it, and opened it in HM3 - all went fine and I can see it is identical to the database I'd copy/pasted. Now I'm on the step of purging older hands from this one to make it my smaller current/play database, but when I'm trying to purge hands I'm hitting an issue where the progress reaches 85% slowly but at around 1% every 10 seconds but goes no further once at 85% (after waiting it out for an hour). It wasn't marked as not responding in task manager, and was still using just under 6gb of ram, 14% CPU, and 0-0.1MB of Disk, but no progress for an hour. I therefore tried to cancel, which didn't do anything, so I had to quit from task manager. I then in Windows deleted that database in case half a purge operation might have screwed it up some, and again made a fresh copy of the big one and renamed it, and opened it in HM3. Again, it was fine and identical to the original. This time I thought I'd try to 'help the purge along' by having it bite a smaller chunk to purge, so I selected a shorter timeframe (one year) to purge, but the exact same thing happened: 1%-84% at not fast but watchable speed, then an hour sat at 85%. It seems odd that with two different timeframe selections to purge, they both get stuck (or appear to get stuck) at 85%.

    I'm unsure if it's genuinely getting stuck when it hits 85%, or it's just looking like it is and I just need to wait it out (like when Windows 10 Feature Updates get to a certain percentage in the installation progress bar quite rapidly only to then sit there for half an hour without budging).

    I wonder if something specific happens for the last 15% of a purge (regardless of the purge selection size) that is different to what goes on for the preceding 85%, and if that task takes longer to process (or is not functioning for me)? That would explain why both times it has appeared stuck at that same 85% progress point. Perhaps I'm hitting a hard limit of what purge can do and I need quit out again from task manager and retry again this time selecting an even smaller chunk? Any advice appreciated.

    Thanks.

  5. #5
    Junior Member
    Join Date
    Feb 2015
    Posts
    13

    Default

    Lol. Pleasant surprise, right after I'd posted that and went back to see the progress, it had completed. So I can answer my own question, the purging progress % is like Windows Feature Update progress bar in that it gets to a point, stops and looks like it's all going pear-shaped, but if you wait it out it completes. Must download more patience
    Last edited by Demonic666; 11-08-2019 at 02:14 PM. Reason: To make sense :)

  6. #6
    *** HM3! *** fozzy71's Avatar
    Join Date
    Jun 2005
    Location
    HM Support
    Posts
    32,803

    Default

    Ugg, I feel dumb now for not mentioning a known bug with purging that is going to affect your plans:

    HMT-3929: Purge Hands does not reduce database hmdb file size
    You may want to wait on building databases via purge until that issue is resolved.

    This issue has been documented in detail and provided to the developers to resolve. If you want to be notified when it is fixed please send an email via the 'Contact Support' link and include "Notify Me When HMT-3929 Is Resolved" in the subject line and body of the email.

    Even when that is fixed, a purged database will never be as small as if the same number of hands imported to a new database, as purging can never reclaim all of the used space. You can create a smaller database manually by importing the recent archived hands but you won't have player notes, etc.

  7. #7
    Junior Member
    Join Date
    Feb 2015
    Posts
    13

    Default

    Good to know, I've cancelled the purge that was going (this time for cash games - which I play far more than tourneys) - this one had been running all night and had still only reached 31% complete, so was looking to be a 4 day affair lol. I'll delete that semi-purged copy of my lifetime database and build a new current one by importing recent archived hands. Should be far quicker and making the current play database smaller and upping performance is the main goal of what I'm doing so it sounds the best option for me. Don't feel dumb, nobody remembers everything all at once! I'm grateful the headsup on it, so thank you.

Similar Threads

  1. Renaming a note that already exists on HUD\Popup
    By Shloogy in forum NoteCaddy
    Replies: 2
    Last Post: 01-10-2015, 09:26 AM
  2. Replies: 7
    Last Post: 03-25-2013, 07:22 PM
  3. Renaming a HUD profile
    By micky in forum Share your HUD configurations
    Replies: 1
    Last Post: 08-17-2012, 12:20 PM
  4. Renaming PostgreSQL DB's ?
    By _Loki_ in forum Manager General
    Replies: 2
    Last Post: 01-21-2010, 09:13 AM
  5. Renaming a Database
    By TPayne in forum Manager General
    Replies: 4
    Last Post: 10-21-2008, 12:14 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •