Not a bug - unnecessary disk activity ?

[horizonx]

Hi,

When Running HP, I noticed constant disk activity.

I ran filemon.exe (similar to process monitor - Process Monitor (http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx))

HMImport.exe communicates with HMHud.exe using small temporary files. Writing each second to the same tiny file, may cause HD track wear over time.

Solution to consider :

Integrate HMImport.exe within HMHud.exe
Use DDE
Use sockets

a cheap work around is to allow people to designate within import setup a target temp directory. In which case it would be possible to select a RAM drive.

Thanks

21036 11:18:45 AM HMImport.exe:5032 QUERY INFORMATION C:\Documents and Settings\user1\Application Data\HEM Data SUCCESS Attributes: D
21037 11:18:45 AM HMImport.exe:5032 OPEN C:\Documents and Settings\user1\Application Data\HEM Data\ SUCCESS Options: Open Directory Access: 00100001
21038 11:18:45 AM HMImport.exe:5032 OPEN C:\Documents and Settings\user1\Application Data\ SUCCESS Options: Open Directory Access: 00100000
21039 11:18:45 AM HMImport.exe:5032 DIRECTORY C:\Documents and Settings\user1\Application Data\HEM Data\ NO SUCH FILE FileBothDirectoryInformation: *.rsh
21040 11:18:45 AM HMImport.exe:5032 CLOSE C:\Documents and Settings\user1\Application Data\HEM Data\ SUCCESS
21041 11:18:45 AM HMImport.exe:5032 QUERY INFORMATION C:\Documents and Settings\user1\Application Data\HEM Data SUCCESS Attributes: D
21042 11:18:45 AM HMImport.exe:5032 OPEN C:\Documents and Settings\user1\Application Data\HEM Data\ SUCCESS Options: Open Directory Access: 00100001
21043 11:18:45 AM HMImport.exe:5032 OPEN C:\Documents and Settings\user1\Application Data\ SUCCESS Options: Open Directory Access: 00100000
21044 11:18:45 AM HMImport.exe:5032 DIRECTORY C:\Documents and Settings\user1\Application Data\HEM Data\ NO SUCH FILE FileBothDirectoryInformation: *.pft
21045 11:18:45 AM HMImport.exe:5032 CLOSE C:\Documents and Settings\user1\Application Data\HEM Data\ SUCCESS
21046 11:18:45 AM HMImport.exe:5032 QUERY INFORMATION C:\Program Files\RVG Software\Holdem Manager\Importing\Status\AutoImport.Reload NOT FOUND Attributes: Error
21047 11:18:45 AM HMImport.exe:5032 QUERY INFORMATION C:\Program Files\RVG Software\Holdem Manager\Importing\Status\PlayerFilter.Reload NOT FOUND Attributes: Error
21048 11:18:45 AM HMImport.exe:5032 QUERY INFORMATION C:\Program Files\RVG Software\Holdem Manager\Importing\Status\Close.dat NOT FOUND Attributes: Error
21049 11:18:45 AM HMImport.exe:5032 QUERY INFORMATION C:\Program Files\RVG Software\Holdem Manager\Importing\Status\Command.dat NOT FOUND Attributes: Error

[Patvs]

What sort of computer do you have?
I've never had more than 20% CPU activity caused by HMImport.exe only during massive imports.


Read:
http://forums.holdemmanager.com/manager-general/11194-top-10-hem-performance-increase-tips.html

to see if you can do other things to increase performance.

[horizonx]

Filemon monitors Hard drive activity. To make it simple HM generates a lot of disk churning. This has nothing to do with CPU activity.

This is because the importer and HM communicates through temp files instead of RAM.

The importer looks into the Pokerstars log files then writes the hands on to the temp files.

HM looks every second or so for those files which may or may not exist, depending if new hands were completed.

Each time this look up goes on, boum, boum, boum, disk / directory seek.

Every time a new hand is played, boum, boum, boum the importer writes to the very same files, squeek, squeek, squeek, the same HD tracks get writen over and over.

Over the long hall those tracks will most likely fail.

The very least that could be done is to allow us to select the temporary directory so we could use a RAM drive for those temp files.

I think a HM programmer should be able to confirm my analysis on how HM and importer work together.

The difference with the importer looking for new hands in the Pokerstars hand log is that the PS file is additive so new tracks are used all the time as the file grows up. That's very different from writing to the same tiny files over and over.

Any sound programming method should minimize disk activity.

Thanks

[Veteran68]

Not that I think this is a big issue from a drive "wear" perspective -- disk I/O is constant in Windows from a variety of processes and the small pulses from HEM are likely not going to degrade disk life to any noticeable degree (besides, HDD tracks don't "wear" in the conventional sense as there is no physical contact, and it takes many tens or hundreds of millions of writes to magnetically degrade disk media).

That said, I'm surprised that common files are used for this sort of interprocess communication. I would agree from a performance standpoint that using disk I/O to communicate between two processes at this frequency is woefully inefficient. There are all sorts of IPC methods that would be much preferable. RPC, DDE, OLE (not that I'm a fan of it), sockets, shared memory, and memory mapped files are just a few examples. File I/O can have a huge impact on system performance without measuring a blip on CPU activity. When we hear of people's systems bogging down while massively multitabling with HEM+HUD and software like TableNinja, it's more likely for disk I/O reasons (or lack of sufficient RAM, which generates virtual memory disk I/O), than for CPU utilization.

[roy.goncalves]

Hi Horizon,

Just to clarify the files that are being created are not to allow HmImport and HoldemManager to communicate but rather HmImport and the hud when live hands are being played - I think you know that but wanted to mention it

The amount of disk activity will not have any real effect on the hard drive. Each file represents the stats for a single table so it would be updated about once every 30 seconds for the x number of tables you have open.

Having the option to specify the temp folder is an option if you can convince us that this is a real issue since of course dev and testing time for this would take away from dev and testing time for other features. If it is a real issue then of course we would be open to making the changes.

The reason we stayed away from DDE and Sockets for HM was that in previous projects I've worked on where the install base was very large there were inevitably issues with DDE and Sockets on some systems that were very difficult to diagnose vs file based systems which were pretty much fool proof. Additionally, it allows 3rd party programs to easily tap into the table data.

That all being said, we have been looking at different ways to do this and evaluating whether they are an improvement over what we have now.

Roy

ps I created a new forum account but it really is me :)