Top 10: HEM performance increase tips

[Patvs]

**If you are using HM2 please see this thread.**

I just thought it would be helpful to post this list. The HEM FAQ has most of this information, but it's quite cluttered to find it in so many FAQs.

Please post here, if you can think of other tips that will help increase the performance of HEM, HM Import, Table Scanner, HUD and postgreSQL. Specifically for those that have a HUGE database. Ask questions, and HEM users can discuss about their experiences using these tips here.


Top 10 tips:

Upgrade your HARDWARE*:
1 to 3

-1 Get a fast harddisk! (= more important than having a QuadCore over a DualCore processor!)
---x Preferably a fast SSD harddisk (fastest) for your database and Windows (Intel or OCZ Vertex 2/3)
---x A new 5400/7200 RPM IDE/SATA harddisk (slowest)
The SSD not only wins on Sequential Read/Write speeds (of big files), they win the most with access time, 4K Read/Write speeds (small files) and just "overall" I/O operations per second.
A Western Digital VelociRaptor used to be easily twice as fast as the best 7200 RPM harddisk. However it currently only is about 10% faster than the latest Seagate Barracudas, Hitachi Deskstars, Western Digital Caviar Blacks and Samsung Spinpoints F1/F3. The VelociRaptor does not justify its price difference anymore.

Note: a brand new $60 high-end 2012 model 7200 RPM harddisk will be twice as fast as your 3-4+ yrs old 7200 RPM harddisk!...

LAPTOP: ALL 5400 RPM laptop harddisks are slow (favoring being energy efficient over performance. Try to find one with a SSD.
-make sure the Control Panel-->Power Options are set to HIGH PERFORMANCE on your computer
-Expert Users only: overclock your CPU!

An alternative to a SSD SATA harddisk: are the new PCI-Express cards like the OCZ RevoDrive. And if you have a USB 3.0 mainboard the SuperTalent RAIDDrive)

-2 Get lots of RAM (at least 2+ GB RAM recommended if you have a 10+ GB database) (note: when using more than 3 GB RAM, use a 64-bit version of Windows)
-3 PostgreSQL loves multiple CPUs, so the more CPUs the better. (Intel i5 / i7 or AMD Phenom) Also "Holdem Manager will be adding improvements that will take advantage of quad core/64 bit versions."

-Change Processes Priority. Start the Windows Task Manager, processes tab--> and while running an auto import in HoldemManager---> rightclick on HoldemManager.exe, HMHud.exe, HMimport.exe---> set priority to HIGH (not real time!), set affinity to use all your CPUs. (click for more detailed info)

Update your SOFTWARE:

-4 Always use the latest version of Holdem Manager, and use the most current version of PostgreSQL. We support 8.4.x by default.

Advanced Users: You are allowed to use 9.0/9.2 (64 bit). They will work just fine*, but they're not officially supported. *Some features (like using the internal backup/restore tool may not work. You can manually use pgadminIII instead: http://faq.holdemmanager.com/questio...estoreDatabase (this is for HM1!)

How to install HEM + PostgreSQL 8.4.
Update your Windows (use Microsoft Update, and make sure you have the latest Service Pack (SP2 for Vista, SP3 for XP)) and update all system drivers (including your mainboard and videocard driver! and .net framework updates).


-5 Remove the WinnerName, WinnerCards and WinnerWon from the Hands View Column Selector

-6 Defrag your non-SSD harddisk, using Defraggler by Piriform. Or you can use Disk Defrag by Auslogics or Power Defragmenter (all FREE software and better than the Windows included Defrag)
It is recommended to stop postgreSQL from running before you use defrag! (To stop and start the service: Start > All Programs > PostgreSQL > Stop Service / Start Service)

PostgreSQL:

-7 Edit your PostgreSQL.conf file in the SQL 8.x/data folder. (you need to reboot after you've made the changes)
Do not use the Tuning Wizard. On some systems it will make changes to the postgresql.conf it shouldn't, resulting in being unable to connect to the database!
If you experience this problem--> simply go to the 8.4/data folder, undo the changes you made and reboot.
Recommended changes:
shared_buffers = <number> MB (set it to 20% of your system's memory, but not higher than 512 MB)
effective_cache_size = <number> MB (set this to 4x the size of the shared_buffers)

Change at your own risk: (change them back to default if postgresql stops running, or your performance decreases)
wal_buffers = 8MB
temp_buffers = 24MB #(wal_buffers x3)
work_mem = 128MB
maintenance_work_mem = 512MB
commit_delay = 10000
checkpoint_segments = 128
checkpoint_completion_target = 0.9


Remove the # at the beginning of the line after making the changes you want otherwise PostgreSQL will still ignore them (lines with a # at the front are not interpreted).


Decrease the SIZE of your database:

-8 How to optimize the database size in 3 steps: -1/2 Vacuum + Analyse, -3 Reindex
--8.1/8.2 VACUUM + Analyse: VACUUM Analyse your database regularly (How to do a manual VACUUM)
--8.3 REINDEX (see 8.1 for instructions, but choose Reindex instead of Vacuum in pgAdmin)

-9 Purge hand histories (It will free up disk space without losing stats, but you lose the option to see the HH or view the hand in the replayer). This is only for HM1! In HM2 a purge deletes the stats too.

-10 How to make PostgreSQL stop making those big log files (I had 8 GB of logfiles before I realized you're allowed to delete the content of the data/pg_log directory!)

-11
--11.1 A near-empty harddisk is faster than a near-full harddisk, so delete your crap! CrapCleaner by Piriform and Windows Disk Cleanup can help.
--11.2 Turn off indexing and turn off "compress this drive to save disk space" in your harddisk properties.
--11.3 Optimize Windows for "best performance" (in Control Panel/System/Advanced/Performance/Setting/Visual Effects).
--11.4 Advanced users only: save space by turning off System Restore (System/System Protection). *not recommended*

-12 Windows 7 + SSD
Windows 7 users, General Windows 7 Performance Tweaks guide*
(If you already have a SSD harddisk also do all the SSD Tweaks in that guide.. though don't disable hibernation!, don't completely disable your page file!, and don't disable AERO!)
*Services to disable in that guide: also don't disable Secondary Logon, Hibernation, and SuperFetch.

Anyone can think of #13 and other tips?</number></number>

[morny]

Thanks for this, were updating the FAQs at the moment so ill give this a few days and see if theres any more suggestions and ill forward it to the guy we have updating the FAQ, if it drops down over the next week feel free to bump it to remind me to add it

[mksmipalla]

Good list. All important things. On number 6 I recommend using Defragler I have had good luck with it overall and does a nice job with postgre. Only thing I would add is that it seems to work better if you stop Postgre from running before you defrag. If your not sure how here is a few pics.

Open your control panel, then click on "Administrative tools"




After opening tools then click on " Services" and the service window will open with all of the running services. Scroll down until you see PostgreSQL and click on it.



Now just click on the " Stop the service" and you will see it change to "Stopped" in the Status list.

Then perform your defrag. After it's done just come back and click on "Restart the service". Done

[morny]

A quicker way to stop and start the service is Start > All Programs > PostgreSQL > Stop Service, either works though

[Patvs]

Thanks for this, were updating the FAQs at the moment so ill give this a few days and see if theres any more suggestions and ill forward it to the guy we have updating the FAQ, if it drops down over the next week feel free to bump it to remind me to add it

bump... what is the status of the updated FAQs?

[fozzy71]

They are still being worked on. Morny no longer works for HM. I have taken over his forum responsibilities, and I believe Mike has taken on the bulk of the responsibility for the FAQs. I am starting work on re-writing one of them now, to try and lighten Mike's load. I have no way of knowing an ETA at this point. Mike might be better informed, but not sure if he could give an accurate ETA either yet.

[All Apologies]

Curious how often should someone do these to free up space?

[pr0wler]

I'm really confused how to do the vacuum and stuff. It keeps saying to do these commands...but where/how? Do you just login to postgreSQL? If so how do you do that?

[PCP Poker]

How to optimize PostgreSQL database size (I haven't tried the reindexing and dump/restore. Can anyone confirm it's save to do that to your HEM database?)



Anyone can think of #11 and other tips?

I habitually do the dump/restore. It cuts the db size by at least 70% on average and greatly improves performance. Sometimes it just doesn't work though. You can do everything perfect, and it just doesn't work. This is one of the reasons i'm really starting to hate postgre. Stuff always decides just not to work. It's like a built in AI that know how to mess with you....

[Patvs]

I tried reindex (for my whole database). The size decreased from 16.4 GB to 15.8 GB (only 4%, but still a useful 600 MB more free space and smaller database)

The two greatest posts on the forum I found this week, were -1 the post by mksmipalla (Want to stop Postgre from creating those large log files, here's how). Morny added a link to that post in the FAQ. I noticed every FAQ entry has tags. For example, this FAQ entry has: "pg_log postgreSQL large log files save space reduce logs postgresql"

However when I click on "PostgreSQL" to see ALL the FAQ entries with that tag, it shows me:

15 FAQs Found
Page 1 of 2

It shows 10 FAQ entries on page 1. And I'm unable to view page 2! (where I'm guessing this new entry is)
I tried different browsers. The Go to Page 2 button is javascript, but it doesn't seem to work. (does it work for you?) Since I've been using the tags mostly to browse through the FAQ it seems this is the main problem why there are some posts I know that exist, I just cannot find.

(-second great post I found was how to change the colors of all the lines in the graphs, including the EV line, hopefully that will be back in the FAQ soon)

----

Also I was hoping a similar post like this one (Top 10: HEM performance increase tips) could be made about Backuping-Restoring a Database / Migrating a Database to a New Computer-Harddisk / Backuping your HEM settings (ALL of those combined in one simple overview, using a similar layout as this topic) since those are currently spread all over the FAQ in seperate topics and I would need all the info if I wanted to re-install HEM on a new computer.

And the FORUM could benefit from a general Anthology Thread topic listing the most usefull forum posts by category.


I'll update the main post as much as possible with all of your input.