PDA

View Full Version : Slow Import



Kaufi
09-30-2008, 05:02 AM
When i started using HM i had an average import over 80 hands/s.

Since few days my computer cant import more than 20 hands per second.

Should i reinstall postgre? and HM? and yes i vacuumed my database :)



Im using vista @ Q6600 2gb Ram

morny
09-30-2008, 08:55 AM
Vacumming is more helpful for improving how quick queries are run but Defragmenting has a better impact on import speeds. Go to Start > Programs > Accessories > System Tools and Run Disk Defrag

Your import speed will decrease somewhat anyway as your database becomes larger but hopefully the defrag will improve things.

Kaufi
09-30-2008, 09:13 AM
thanks ill go for it soon :) im still importing hands gg, ill report when it helps

ActionZzZ
09-30-2008, 10:20 AM
Vacumming is more helpful for improving how quick queries are run but Defragmenting has a better impact on import speeds. Go to Start > Programs > Accessories > System Tools and Run Disk Defrag

Your import speed will decrease somewhat anyway as your database becomes larger but hopefully the defrag will improve things.

I made Defrag and Vacuum, but speed import still very slow (20h\s) :(
PC: 2.1Ghz, 1G Memory, Win XP

in FAQ you write:
Q: What is a "normal" import speed?
A: It really does depend but on a 2 ghz desktop you should be able to get close to 70 hands per second for 6max cash games or about 250K hands per hour!!! On a Dell M1710 laptop I am consistently able to import at over 100 hands per second. Tournies and Full Ring do import quite a bit slower since there is so much more data (ie more players).

HOW to receive 70h\s ?


PS I bought HM for fast importation of hands...

DocBru
09-30-2008, 10:27 AM
I have the same problem... currently importing at about 30h/s with 1.06.03h, had about 80-100 with earlier versions

Kaufi
09-30-2008, 11:47 AM
wooohooo after defrag HM boosted the import up to 5!!!!!! hands per second ... lol :D shouldnt it boost the import? gg may the time will bring some changes

fabio
09-30-2008, 02:21 PM
wooohooo after defrag HM boosted the import up to 5!!!!!! hands per second ... lol :D shouldnt it boost the import? gg may the time will bring some changes


You need to stop the PostgreSQL service and close all postgres.exe before defragging

Kaufi
09-30-2008, 03:48 PM
i did as said but no big change :p <30hands/s another solution anyone?

Diesel7
09-30-2008, 08:24 PM
Don't use DEFAULT WINDOW's defragmentation program.
I advise using O&O Defrag in Access Mode. Super efficient!

P.S. After having more than a million hands in my DB, i still keep importing speed up to 50hands/sec!

Kaufi
09-30-2008, 08:44 PM
now im defragging my disc with auslogics disk defrag :p its a freeware program and it got a lot of good feedback. so go on and hope your help give me more than 40hands per second :p

i need to import 300k hands daily and thats hard with 20h/s...



nope now its @ 17,6h/s ... where can i find how many hands ive in my db? ... maybe ive already to much for a good performance

Diesel7
09-30-2008, 09:04 PM
Yours Q6600 2gb Ram on Vista vs my Core Duo 3Mhz 4gb Ram on XP should work with the same speed. My computer is fresh-installed, setup pure for poker, oh couple of games, that's it. I import twice less amount/day. So i guess you can hope for 30-35hands/sec IF you do this:
1)import
2)vacuum/analyze
3)defrag O&O or Auslogic
4)clean registries with C&C, Registry First Aid or something powerfull
5) Once in 10 days clean PostgreSQL/data/pg_log folder.
Oh, and do backup your system, of course.
GL.

P.S. Actually i am wrong, your Q6600 on Vista with 2Gb Ram is slower than my setup, so that's probably right 20 hands/sec with the way Hem works with Postgre right now, but doing those steps should increase your import by 5-10hands/sec. I hope Hem will improve about this issue by the end of the year.

Kaufi
09-30-2008, 09:19 PM
the first million hands i had over 80 per second :)

the problem is... when i look at the pc usage... CPU usage @ 16% and 1,3gb ram are in use... so i think HM dont use the full speed of my pc

but i will follow your steps! thanks!

Diesel7
10-01-2008, 04:38 AM
now I've got 34 hands/sec speed.
P.S. brag - it went up to 43 hands/sec, nice:)
P.P.S 47.3 hands/sec at the end of import.

Q to HEM team:
right now I am importing hands and this stats of resources shows up:
Usage of disk C:
HMIMPORT up to 456K/s
Postgre up to 5,500K/s
second postgre up to 2,500K/s
third ~ up to 188K/s
fourth stays around 42K/s

Processor usage:
total CPU usage up to 28%
HMIMPORT up to 53%
Postgre 3-30%

memory usage is lower than 26% (922MB used out of 3582MB)
So, my Q is: are you guys planning on improving the communication of HEM with PostgreSQL?

morny
10-01-2008, 03:25 PM
Yeah we will be and Roy actually did make alot of improvements a while back but sometimes people using similar setups are getting 2-3 times the speed of others so its a tricky area to figure out why it works with one so well and not another. Anyway Roy said hed look into tweaking this again in the future but is very busy at the moment so its just finding the time to fit this in without letting down people waiting on other improvements

Diesel7
10-01-2008, 08:32 PM
ok, good, this is not main priority for sure.
Well, 2 dyas of trial left and so far i am 85% satisfied with HEM and time to pruchase. looking forward to great improvements, hopefully in 4-6 months it will be done. Gl, guys.

Diesel7
10-06-2008, 03:46 PM
the bigger my DB gets the faster i run import. Now my speed is 120 hands/sec for last 2 days. ROFLMAO

cantona
10-07-2008, 10:41 AM
Ecxecpt mine are pittyful.

The first 30K hands imported at 18h/s, (FR hands). Which was so so ok. Thereafter it soon steadily decreased to now a woeful 5-7 range. I now have just under 500K hands.

I have read the comments re: defrag and that is likely to be ok as done recently, but firstly I must Vacuum the DB. In HEM it says there is a problem with DB and I have to do it from PgAdmin. I cannot find an option in Pg Admin to do it?

Can someone help please?

morny
10-07-2008, 11:42 AM
http://208.109.95.123/faq/afmviewfaq.aspx?faqid=163

cantona
10-07-2008, 06:11 PM
I have completed the vacuum, and carried out a defrag, and this produced only a slight increase in performance, now runs at 9h/s after one test.

Is there any thing else to try?

morny
10-07-2008, 06:16 PM
Not really, most people have good speeds but some people are still experiencing slower speeds like yourself, Roy has already said hed come back to this as soon as he has some free time and work on tweaking it

Northernwind
11-21-2008, 05:45 PM
I found interesting trick to faster import, restarting 3-4 times import brings speed up to 3-4 times from 1st attempt :)