PDA

View Full Version : importing mass hands



absolid
04-02-2010, 07:51 PM
I updated my computer recently to make this process faster, quad core cpu, 4GB ram and 2x seagate 7200rpm 500GB drives in RAID-0. I know SSD would of been better but I had to save that for the next upgrade.
Anyway I started importing a 1 million hand batch and was getting 80+ h/s, which I knew was too good to be true as I've read around 30h/s is a good figure. By the 100k mark it was still doing 50+ h/s. I went to bed and when I woke up it was only doing 6h/s. I vacuum analized, reindexed, and got it back to 35 - 36h/s, which I am happy with. My question is: When importing such large amounts of hands is it better to break them down into smaller batches 50k - 100k and then import?
My second question is about the defrag. Can I defrag my disks even though they are running in RAID-0?

Patvs
04-02-2010, 09:05 PM
100k batches = FASTER
defrag RAID0 = YES, please do defrag

Read the top 10 performance tips thread in my signature.
Especially take a look at the power settings of your computer, and edit postgresql.conf

absolid
04-03-2010, 12:32 AM
Thanks Patvs. I have read these before when I had old computer but I will check it out again. This is so strange all these programs, I wish I understood them better. Last night like I said I imported heaps of hands. When I woke up it had done about 220k - 250k of the 1 million and was importing at about 6h/s. This import was basically on new hard drives and new data base. Anyway today I vacuum analyzed and reindexed and now I'm getting 40h/s and was still getting these figures after 200k+ hands.
I just don't get why last night it slowed down so much being that there was actually less in the data base. Just doesn't make any sense to me.
So yes I will do the performance tips and defrag and see what happens, although I am extremely happy with 40h/s anyway.