Moving 2 years of HH to old XP desktop takes tons of time / HD Space

[Scotty12]

Hi guys,

On monday my laptop died mid session, so I switched to my trusty windows XP desktop, updated to 1.11.06a and emailed stars and ftp for my last 2 years of HH. My 500 GB HD was approx half full @ the time.

FTP sent me 33 RAR files that extracted into 4 large (300 MB + ) textfiles
Stars sent a dozen emails extracted to 20MB textfiles. Before bed on monday I created a new DB, named it 2011 and started an overnight import of all the FTP files. It seemed to go smoothly, and the next day I put the first 8 stars text files (128 MB total) in my HH directory and began another import.

This import has been running uninterrupted for 2 days - the import progress shows a green progress bar that appeared to be 100% when I went to bed, and still looks that way when I woke up .. but it keeps displaying the message
'X of X parts of large file have been processed...' where both values are constantly changing. See photo below:



I just went to defragment to see if that might help the problem, and there was only 60 GB of 500 GB free on my HD, not enough to run the defragmenter. Does the 200GB of space eaten by this import signify a problem? If so, how can I correct it? Should I uninstall HEM and all of my old databases and start a new import from scratch?

Edit: After posting this thread I clicked on 'stop import' and it is still 'stopping ...' over an hour later. Here is a screenshot of the task manager:



Thanks

Scott

[Patvs]

Make sure logging is turned off and delete your PostgreSQL logging files, read:
http://forums.holdemmanager.com/mana...ease-tips.html

Update to http://www.holdemmanager.com/Downloa...Internal-1.exe


Just kill the holdemmanager.exe process (if it's stuck at "stopping")
And reimport the rest of the files.
Since the files are so big (300 MB) and they import in parts, importing one 300 MB file should take HOURS and HOURS.

[Scotty12]

Hi Pat,

Thanks for your response. I will upgrade , follow that top 10 and maybe even buy a SSD if I have time this weekend.

The import finally stopped, and it says there were 4.9 M duplicate hands:



This was suprising to me because I havent played close to that # of hands lifetime. When I go into my 'select player' screen it shows me @ 100k hands on ftp and 300k on stars - Any ideas on how / why 4.9M duplicate hands occurred?

Should I try to reimport the files into this '2011' database I created, or just delete my old databases and start fresh? I wouldnt mind clearing up the 200GB of HD space I clogged up these past couple days.

If I do choose to delete the DB and start fresh - is uninstalling HEM the best way to go about doing this? Or do I need to manually delete each DB?

Thanks again

Scott

[Patvs]

SSD won't really help if the .txt files are 20 MB / 300 MB.
It'll be faster... but it will still take hours.

4.9 million hands... eh.... hmmm... you imported all these requested handhistory files into a NEW empty database, right? Not in an existing database that already contained most of your hands?

Maybe FullTilt/PokerStars made a mistake and every hand is listed multiple times in the files they sent you?

The 200 GB database... is it mostly log files that you can delete?
Also edit your postgresql.conf file to assign more RAM to PostgreSQL.
Just reimport the rest of the files into this database.

When you've imported ALL files:
create a .backup of the database! (in pgadminIII)

You don't have to uninstall HoldemManager.

If you want to create a NEW database. You can delete the existing database in pgadminIII, read: http://faq.holdemmanager.com/questio...elete+Database

[Scotty12]

Hi again Pat,

I think I solved the problem by accident doing a 'vacuum analyze' database or whatever its called just now.

After selecting 'standard' and then clicking 'optomize'

[3/4/2011 8:13 PM] Reading databases . . . . .done
[3/4/2011 8:15 PM] Starting optimization
[3/4/2011 8:15 PM] Optimizing 2011...done


My HD instantly showed 235 GB free again. I have yet to do anything with the log files, Im assuming the maintenance just cleared these log files for me?

[Patvs]

Yes.. beside the log files... it's possible there were 200 GB of temporary files on your computer. (.temp files are created when you start but don't finish a .backup... or a vacuum/reindex)

Usually finishing the vacuum/reindex... or even just REBOOTING your computer will delete them.