Very slow importing

[Russell99]

I just got holdem manager and have been importing all my hands from PT and it has been going really slowly. The first 50k hands imported at about 20-30 hands/s, but since then, I have imported about 300k hands in 50k hand chunks at a speed of about 5 hands/s. My last import averaged about 2 hands/s. I am not running anything else while I am importing so I am not sure what the problem is.

Another problem I have (possibly related) is that when I try to vacuum analyze the database a message pops saying there was an error and to manually vacuum my db through pgadmin. How do I do this?

Everything else with the product seems great so far, so it would be nice to get these problems fixed.

Thanks

[morny]

Are you importing full ring, tourneys or 6max/headsup. I know that as the indexes get bigger it takes longer and longer to update them which will result in reduced import speeds however it should level off. It sounds like its a bit slow though at those numbers, what version have you and what is your machine spec.

I know some people had an issue with one of the 1.06 versions where a bug caused import speeds to slow alot but it didnt happen to everyone. The latest release 1.06e was supposed to have fixed/improved this problem so i suggest you upload to that version and see if it has improved and if not let us know.

That error is when vacumming is due to a setting in the postgreSQL folder and is basically a bug which Roy is looking for a workaround at the moment however in the meantime you can do a a manual vacuum if you follow the guide attached:

[thesantini]

I just completed the vaccum as suggested by your help file and the time required to import the hands has increased nearly three-fold.

[thesantini]

Does running the vaccum from HEM accomplish the same?

I'm going to give it a shot since manually vaccuming did not accomplish a faster database import.

[thesantini]

The vaccum through HEM helped a little.

[thesantini]

Is importing half the files to a database an option and merging the second to the first a solution?

Also, I'm importing from a directory of 1200 files, will HEM remember where I am?

[morny]

Im not too sure on this but IIRC vacuum dosent do a whole lot for import speed, its more to improve HUD speeds and i doubt very much it makes a difference whether you do it from PG Admin or HM its essentially the same process just Roy made it easier to do from within HM for people.

How do you mean will HM remember where you are?

[guitarizt]

I've vacuumed my database twice and I'm using 1.06.0e and it's still going at .3 hands a second when I autoimport and datamine. I think it goes slower as the hh files get bigger.

[Rvg72]

Don't worry too much about auto import speeds since most of the time is spent reading the files. I will be working on a solution to make sure that even with large files it does not slow down too much while auto importing.

For the initial issue with very slow import speeds, what version are you using? If it is not 1.06.0e then try downloading www.rvgsoftware.com/hm/hm1.6.exe

Also, I have heard from some people that defragging their drives has actually doubled or better their import speed - obviously depends how fragmented your drive is. One more thing, larger files import faster, especially with 1.06.0e so if you can get your hand history files to be about 1MB each that should produce the best speeds. I am working on overall improvements along with specific improvements for small files.

Roy