Importing database slowing down
Page 1 of 2 12 LastLast
Results 1 to 10 of 12
  1. #1
    Junior Member
    Join Date
    Nov 2009
    Posts
    25

    Default Importing database problems

    I'm in the process of transitioning from hm1 to hm2 and importing the whole hm1 database into hm2 at the moment.
    Using a fast modern laptop with SSD drive.
    The database consists of 3.3 million hands and it imports 335 hands/s. Went really smooth the first 1.4m hands, after that it still says 180-330 hands/s but "Importing XXX.XXX of 3300000 hands" where X is moving extremely slow (maybe 100 hands i 5 sec). Is this normal that the process slows down? Think this is going to take forever (days).

    Update:
    The import was finished by 1.5m and I got a message that there was an error importing the HUD and that i should try prefs.xml. Tried to import the hud manually in the "hud designer -> import 1.0" now, the same error message, won't import my hud.
    Checked the total amount of hands in my new hm2 database and it contains only 1.339m hands out of 3.3m?
    Last edited by criztal; 12-24-2013 at 02:05 PM.

  2. #2
    HM Support Patvs's Avatar
    Join Date
    Feb 2009
    Location
    Amsterdam, the Netherlands
    Posts
    33,523

    Default

    It's normal the process slows down.

    Do you also have all the original handhistory files of the 3.3 million hands?
    Instead of importing a 'HM1 database'.

    It's faster/saver to import all the original handhistory files into HM2.
    If you don't have those files (in your HM1 archive folder), export them from HM1: Options--> Database Management--> Connect--> select your database--> export.

    The advantage of this method is, you can import -let's say- 1 million hands. Stop the import process. (before it gets extremely slow)
    Vacuum/Analyse, Reindex the database in pgadminIII, then import the next 1 million hands.
    Participate in the Beta release of the newest Hold'em Manager version: HM Cloud. Sign-up HERE.

    If you would like to leave some feedback to help us improve the quality of the solutions, and/or the support quality you received, - you can do this here

  3. #3
    Junior Member
    Join Date
    Nov 2009
    Posts
    25

    Default

    Thank you for your answer.
    I have alot (not all) original Handhistories saved in a folder but tried the way you described it and it was no problem. But I'm running the whole import now for the third time from start to finish and every time the database is finding new hands to import and alot of duplicates obviously and some errors (which is normal i presume). I guess i'm gonna do the import a couple of times until there are no new hands imported? And try to import the original handhistories on top of that just to be sure?

    Another question: When i imported the whole hm1 database but only got about 1/3 of the total hands, can i be sure that all notes were imported aswell or just partially?

  4. #4
    You're out! udbrky's Avatar
    Join Date
    Jul 2008
    Posts
    20,898

    Default

    Had you purged any hands in HM1? Sounds like the DB may be corrupted.

    1) Please close HM

    2) Please open PGAdminIII

    It is in the Start -> Postgres folder.

    3) Enter your postgres password. The default password is typically postgrespass.

    4) Open the databases.

    5) Highlight your database.

    6) Right click, click maintenance.

    7) Run the vacuum and analyze and reindex functions.

    http://faq.holdemmanager.com/questio...lyze+Database+

    Please try again.

    Did you import the prefs file from hm1?
    Regards udbrky (Chris)

  5. #5
    Junior Member
    Join Date
    Nov 2009
    Posts
    25

    Default

    I haven't purged any hands in my hm1 database that i can recall except one or two just to try one time. But there are some hands from "svenska spel" (Boss) which gives some incomplete/error hands from time to time for me and for other users of the site that i'm aware of.

    Here is what i've done:
    I tried to import all the hands from the original hand histories by importing from folder. Added maybe 150k of hands and a ton of import errors (over 800). I ran a Maintainace/Full Vacuum/Analyze/Reindex succesfully on my new hm2 database and then imported the hands i recently exported from the hm1 database as you suggested once again and like before, it keeps adding new hands everytime i run it. I checked the total amout of hands in the newly created hm2 database and it has now grown to almost 3.1 million.

    When i tried to import the whole hm1 database in the beginning i think the program tried to import "prefs.xml" but i got this message: "Error reading HUD XML file. Are you sure this was the right one? Try prefs.xml" and yes i have tried to import "prefs.xml" in the import 1.0 hud but i always get the same error message so it wont let me. (i did however export my hm1 huds and imported them manually into my hm2 database)

    Could you please answer my question about the notes aswell?

  6. #6
    *** HM3! *** fozzy71's Avatar
    Join Date
    Jun 2005
    Location
    HM Support
    Posts
    32,803

    Default

    To avoid the duplicate hand history issue slowing down the import process in the future you need to section off the exported hands from HM1 into multiple folders. Then you can import 1 batch now and when you import the next batch there shouldn't be any duplicates. For the error hands please open the HM2 > Hand Importing > Import History menu and Show Errors > Save All As... and then zip and email the import errors for us to look at. Please zip and email the problem hand histories, with a link to this thread and your forum name, to hm2support@holdemmanager.com

    I am not sure if all of the notes will have made it into the new database if you canceled the import from HM1 process. I am not sure if they are imported at the beginning, end, or throughout the process. The only way to be sure would be to export the notes from HM1 and create a backup of just the Notes in HM2 and then extract the playernotes.xml file from the HM2 backup and compare it to the HM1 exportednotes.txt file in a text editor but this will be a tedious and laborious task. You could also try checking some of the players in both HM1 and HM2 at the same time in the replayer HUD.

  7. #7
    Junior Member
    Join Date
    Nov 2009
    Posts
    25

    Default

    But the duplicate hands aren't imported into hm2 i presume? So that's not that big of a problem since the imports are pretty fast with my SSD drive. I just don't get why "import from folder" is finding new hands to import everytime i run the import on the same folder (exported hands from hm1 database).

    The import from database was not canceled however, the process finished but stopped after 1/3 of the total hands. Thanks for the tip, i'm going to try it.

  8. #8
    *** HM3! *** fozzy71's Avatar
    Join Date
    Jun 2005
    Location
    HM Support
    Posts
    32,803

    Default

    Yes, duplicates are skipped/ignored. If you let the import run all the way thru on the exported hands folder it should find all the hands and if you were to run it again later it would all be duplicates as long as you haven't added any new files/hands to the folder. If the import from database finished but was missing some hands then hopefully all of the notes were imported. The notes are separate from the hand histories (they are part of the database) so they should have been imported completely hopefully.

  9. #9
    Junior Member
    Join Date
    Nov 2009
    Posts
    25

    Default

    Quote Originally Posted by fozzy71 View Post
    If you let the import run all the way thru on the exported hands folder it should find all the hands and if you were to run it again later it would all be duplicates as long as you haven't added any new files/hands to the folder.
    Yes, that is totally logical and what i expected, however the funny thing is, i've made that now numerous times and every run i do on the same untouched folder with my exported hands from the hm1 database, the importer finds new hands to add to the database, not a small number, about 100-200k and the rest duplicates. I just don't get why it's finding new hands from an already imported untouched folder time after time, is something corrupt?

  10. #10
    *** HM3! *** fozzy71's Avatar
    Join Date
    Jun 2005
    Location
    HM Support
    Posts
    32,803

    Default

    Quote Originally Posted by criztal View Post
    Yes, that is totally logical and what i expected, however the funny thing is, i've made that now numerous times and every run i do on the same untouched folder with my exported hands from the hm1 database, the importer finds new hands to add to the database, not a small number, about 100-200k and the rest duplicates. I just don't get why it's finding new hands from an already imported untouched folder time after time, is something corrupt?
    I can not reproduce this so the only thing I can think is you might have a corrupted/damaged database caused by the incomplete Import from HM1 attempt.

    I think you should create a backup of your player\hand notes for this HM2 database and [x] Skip Database then restore that to a new HM2 database and import the hands you exported from HM1 to the new database.

Similar Threads

  1. hem is slowing down
    By AstonMartin in forum General Support
    Replies: 2
    Last Post: 12-18-2013, 07:38 AM
  2. Importing hands from old database to new database?
    By Gaylord in forum Manager General
    Replies: 3
    Last Post: 12-14-2010, 11:05 AM
  3. HM Slowing
    By The Minder in forum Manager General
    Replies: 1
    Last Post: 11-12-2010, 04:46 AM
  4. HEM importing PS HH very slowing - No HUD
    By SuperSized in forum Manager General
    Replies: 5
    Last Post: 02-21-2010, 07:25 AM
  5. HEM slowing down
    By ranka in forum Manager General
    Replies: 3
    Last Post: 08-16-2008, 05:44 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •