So what's the best HD setup for HEM/database?

[Hunter]

Lets assume I don't mind paying up to 1K for the storage component of my computer and I'm trying to optimize it for HEM.

It seems everyone is in agreement that putting your Postgres on a separate drive is optimal?

Now, should that drive be SSD or SATA? From what I gather, SSD is faster for reading, but SATA is faster for writing? Also the SSD drives will wear out faster, but that number is still in years? Which drives in particular should one be looking for?

RAID? It seems most agree a RAID0 configuration would be optimal? If I'm buying a new MOBO w/ RAID do I need a dedicated RAID card? It seems the advantages to a dedicated card are that it off loads CPU processing (which I would guess is not an issue with a high-performance modern processor?) and it provides a battery backed memory unit that increases write times for small files? Are there any other advantages? If I need a RAID card, do you recommend one?

Drive Quantity? Assuming we're doing RAID0, how many drives do we want? Is there a point where we stop seeing performance increases or at least they're marginalized?

32Bit or 64Bit OS? I guess the only advantage to 64Bit is you can throw more memory at it, but assuming you have at least 3GB (+1 on video) does postgres benefit from having more, or rather does it benefit in a noticeable manner?

Budget. Again I'm sure we could go crazy here, but I don't really want to spend more than 1K on storage (drives + controller (if applicable)) unless there's some huge performance increase at $1,100.

Performance. Has anyone done anything like this? What kind of import speeds are possible?

Thanks for your help.

[morny]

Escalated to Roy, hell respond asap

[Bacco777]

bump

[The Minder]

Personal opinion.

As long as you're running Windows all the hardware upgrades in the world won't make a big difference. Sure, there will be some slight improvements, but nothing radical. There's just too much overhead in Windows and as our databses grow the problem will only get worse. There are threads here, and FAQs on the HEM web site telling how to spead things up... good words.

If you've got a $k to spend, I'd recommend:

1. buying a couple of 80Gb, 7200 quality drives... RAID them or not, it's no big deal. Btw, RAID is NOT an effective means of data backup.
2. Reformat your drives to Linux (I use Ubuntu, but pretty well any of the distro's are good) and use this as your primary O/S. Install postgress here.
3. Download and install Virtualbox and reinstall your Windows into this client... leave it as a bare bones O/S, ie no anti-virus etc. Load HEM and your poker clients into this guest.
4. Download/install the ODBC file into the Windows guest from the postgress site.

By setting things up this way, all the hard yacka is being done on the Linux side where things perform better. HEM and your poker clients are free to perform at their max as all the load has been taken from that part of the system.

My 2c worth.

[Patvs]

Performance. Has anyone done anything like this? What kind of import speeds are possible?


Lets start with your final question. I've tried importing 10+ million hands
on my AMD Athlon, 1 GB RAM, 7200 RPM SATA harddrive. It were 100.000+ files (most duplicate files) a couple really big annoying 20+ MB files (from a exported database) and the import would take 50 days (!). My CPU/RAM were both stressed 100%!. (I NEED A NEW COMPUTER)

So I took all my handhistories to a friend with a Q8400 quadcore, 6 GB RAM, fast 7200 RPM SATA harddisk and it would still take 25 days on his computer. His RAM/CPU was ONLY stressed 10-15% (!). His harddisk was the bottleneck.

On the OCZ forum is a PostgreSQL topic (http://www.ocztechnologyforum.com/forum/showthread.php?t=52758&highlight=pokertracker) were they talk about PostgreSQL databases on a OCZ Vertex SSD.
Someone in the topic mentions with PokerTracker he had 330 hands/sec writing to the database with a regular harddisk, with one Vertex he got ~500 hands/sec. But PokerTracker imports hands faster than Holdem Manager.




It seems everyone is in agreement that putting your Postgres on a separate drive is optimal?


It depends. If you install Windows and Holdem Manager on a slow C:system drive, and install Postgres on a fast separate D: database drive... or you have Windows/HEM on a fast C:system drive and install Postgres on a (slow) external NAS drive... then the separate drive makes no sense.

Since the fastest drives at the moment (SSD / Velociraptor, etc.) are relatively expensive, again it makes no sense to buy TWO expensive drives, just to separate your database.




Now, should that drive be SSD or SATA? From what I gather, SSD is faster for reading, but SATA is faster for writing? Also the SSD drives will wear out faster, but that number is still in years? Which drives in particular should one be looking for?


SSD vs SATA (http://www.holdemmanager.net/forum/showthread.php?t=11317&highlight=ssd+harddisk)

The writing speed of a SSD is not that much slower than of a Velociraptor. (and the reading speed is MUCH faster) If you want writing speeds AS fast as a Velociraptor, just put two SSD in RAID-0.

Your SSD WILL wear out. Every block can only be rewritten 10.000 times. So if you have a 60 GB SSD, and you write 300 GB of data to your drive a DAY, if will only last 5 years. Microsoft Windows actually does write a lot of data to your drive (even if you reboot, start a programme, etc.) but not more than 2 GB a day.

The "wear out" problem only is a problem for the first generation notebooks SSDs harddisks, which only had a 10-20 GB capacity, which can wear out faster than in 5 years.


RAID? It seems most agree a RAID0 configuration would be optimal? If I'm buying a new MOBO w/ RAID do I need a dedicated RAID card? It seems the advantages to a dedicated card are that it off loads CPU processing (which I would guess is not an issue with a high-performance modern processor?) and it provides a battery backed memory unit that increases write times for small files? Are there any other advantages? If I need a RAID card, do you recommend one?


You don't need a separate RAID controller, unless you want 4+ drives in RAID or want a RAID5/RAID10 array (or some crazy setup) Or... if your onboard RAID controller doesn't support TRIM.


Drive Quantity? Assuming we're doing RAID0, how many drives do we want? Is there a point where we stop seeing performance increases or at least they're marginalized?


The "problem" with SSDs in RAID0 is, you will always see a performance increase if you add more drives. THE MORE THE BETTER.
So you want 24 SSD drives in RAID0 (you do need a special RAID card for this) Check out this youtube video (http://www.youtube.com/watch?v=96dWOEa4Djs) :)



32Bit or 64Bit OS? I guess the only advantage to 64Bit is you can throw more memory at it, but assuming you have at least 3GB (+1 on video) does postgres benefit from having more, or rather does it benefit in a noticeable manner?


Noticeable... no. But professional databases, configure Postgres to use 12+ GB of RAM (where big parts of the database) are constantly IN the RAM. Of course those database servers wil require a 64bit Windows.



Budget. Again I'm sure we could go crazy here, but I don't really want to spend more than 1K on storage (drives + controller (if applicable)) unless there's some huge performance increase at $1,100.


Unfortunately at $1,100 that's were the FUN starts! With SSD PCI-Express cards. In three months you will find this (http://www.engadget.com/2009/06/06/fatal1ty-teams-with-fusion-io-to-launch-80gb-ioxtreme-pci-expres/) in stores

http://www.blogcdn.com/www.engadget.com/media/2009/06/fusion-io_ioxtreme-ssd-small.jpg

OCZ will release a 1 TB version (http://www.engadget.com/2009/03/05/oczs-z-drive-puts-1tb-of-blazing-ssd-capacity-in-your-pcie-slot/) soon and a 4 TB version at the end of the year.

[Patvs]

Average Read speed / Average Write speeds (in MB/sec)
460/270 2x OCZ Vertex in RAID0
390/270 6 or 8 NORMAL 7200 RPM SATA harddisks in RAID0
230/190 Intel X25-E
230/135 OCZ Vertex
230/70 Intel X25-M
190/180 2x Western Digital Velociraptor in RAID0
140/140 Seagate Cheetah 15K.6 SAS
100/100 Western Digital Velociraptor
90/90 Samsung SpinPoint F1
67/67 Samsung EcoGreen F1


500/400 OCZ PCI-Express / Fusion-IO ioXtreme PCI-Express*
1500/1400 Fusion-IO ioDrive Duo PCI-Express*
2000/2000 24 SSD harddisks in RAID0

*you can't boot to them.. yet.