Bug in importing large amounts a hands.

[Yersh]

When you inport large amounts of hand histories, database index goes broke every 200k-300k of imported hands. These leads to such errors as
"E:\poker\hh\z04.txt ERROR: XX002: index "playerhandsflop_pkey" is not a btree"
or
"E:\poker\!!!hands\titan_sh\z\iPoker_NL100_SH_to_N L400_SH_2008-06-01_to_2008-06-30_36k\Titan_SH_NL100_050708\_TURBO__Santa_Claus_0 5_07_2008.xml ERROR: XX002: index "handhistories_pkey" is not a btree"

Of course you can make a reindex of db, and continue the importing, but when I imported 5M hands, had to make the reindex of db 12 times. That's really annoying.

Also the importing of pokerstars hh is 3 times slower than importing from other sites.

[Rvg72]

Hi, thanks for the info on Stars - will need to look at that obviously.

Regarding the main issue, large imports don't corrupt databases they just make it more likely for corruptions to impact the database. Databases get corrupt due to power shutdowns while something is writing, bad disk sectors from old drives, scandisk on reboots going in and deleting broken file chains (which may have belonged to an index or two thus breaking it), 3rd party disk buffering software that stops the data from writing immediately thereby increasing the chance of corruption etc.

Maybe try installing postgresql on a different, new drive and see if the problems still happen?

[Yersh]

I asked about this problem, because in HM 1.05.09 db went broken only once for 13M of imported hands.