Importing HM1 DB

[Shotfox]

Hi there

I have quite a large DB - nealy 8m hands. I can't leave my computer long enough for this to be done in one go.

Every time I turn HM2 back on the import starts from the beginning and works it ways through all the hands it already imported, recording them as duplicate hands.

Is the only way to do this successfully to use date ranges?

Thanks

Matt

[netsrak]

I suggest you move the already imported files to a different folder which is not automatically accessed by the HM2 import

[Shotfox]

This was not necessary in HM1 - is there any reason that function has been removed?

HM1 will just pick up where it left off if you restart HM at any point. This version seems to want to read all the already imported files again??

[morny]

This should just resume the import if you stop the import and restart it. Try it with a test sample, start an import and close the window when you want to stop it, in the bottom status bar you should see the import progress and a stop button, press that and restart HM2 and then in the same status bar you should be able to resume it. It sounds like you cancelled the import completely and restarted it, in that scenario it would skip over the duplicate files but would still process them nonetheless

[Shotfox]

For example - I imported some hands earlier.

I then added some new hands to the same folder.

I select import from that same folder without moving those already done.

In HM1 the files previously processed would be completely ignored.

In HM2 this is still processing them as duplicate hands before starting on the new hands.

[Sarek]

Which import function you use for this? From folder? Or autoimport (that is wrong)
If from folder (and this folder is hmarchive) - why not to use month-by-month if you don't have much time for all import at once?

[Shotfox]

using import from folder - but not from HM Archive. So unless i go through the folder and split it into subfolders myself its not possible to do month my month etc.

[Sarek]

As I tried, hm wokrs very fast with dublicates
And: do you ever sleeping? PC can live without this...

[udbrky]

Right now, HM2 passes through previously processed hands, and will show that they are duplicates. It is still being worked on to make it like in 1.0, where they are just skipped altogether.

8million hands will take some time. I did 13million in one day - about 24 hours - the other day on a computer with an SSD, quad-core CPU and 16GB ram.

You should import them, let it run, go do something fun, then go to sleep with it importing. The fact is, 8 million hands just takes time to import. HM2 imports 2.5x faster than HM1 and faster than PT3 (my 1million hand sample was the same roughly, but PT3 still had to optimize the DB, it wasn't ready to use, so their h/s wasn't true like HM2 is).

[Daniel]

I'm importing directly from the HM1 database, not via auto import or folders. What does the import do when I quit HM? Does it start over, or skip imported parts of the HM1 DB?? I have 11 mio hands to import, and also not able to do it in 1 go.

Import speed is around 50hands/s for a completely new db.

[udbrky]

It will still read the hh's and see they're duplicates - it won't have to process and parse them, thus making it go through them faster. It's best to let it do as much as possible, or to import from your archive folder and do a year or so at a time, then the next break do the next and so on, if you cannot do it all at once.

This will be fixed in the future though, it will skip over duplicate hands as 1.0 does.

[Daniel]

OK thanks. Does HM2 have a cache rebuilder, so when we purge hands they will be completely removed from the DB including all stats? That would really help to keep a DB small (e.g. only hero hands and 1 year observed HH, remove old observed HH's each year)

[blkmagik98]

I am curious about this as well. If I purge observed hands from HM2, will they still stay in the HUD cache like HM1, or are they gone completely?

[Sarek]

If you purge hands - you also lose stats generated from them

[blkmagik98]

OK, thanks for the reply. If I have to keep the hands instead of being able to purge them like in HM1 (where they would still be in the HUD cache), I will invest in a SSD to compensate for the bigger and slower database.

thanks,

Michael

[Daniel]

If you purge hands - you also lose stats generated from them

Just back from vacation... That's the idea. Too keep the DB small. Stats from other players older than 2 years are probably outdated anyway.

[Sarek]

1) You have "months" setting in HUD Filters
2) You have HH archive. And if your DB contains hands more than 2 years old - possibly is easier to create new one and import fresh HHs only...

[blkmagik98]

OK, thanks for the info. The hand histories that I have are only from the last two months, so I don't want to purge them yet but I can tell the import speed is dropping off quite a bit.

Michael

[Daniel]

Two major concerns which :

- HM2 startup often takes 2 mins. Is this normal?
- HM1 DB import is going nowhere. I tried to import 4 days of HH with observed HH. Maybe a part of this was already imported. Took 5 mins to see 0 out of 3.x mio hands imported. This stays like this without changing or getting the impression of any hands being imported. When I press pause, it says pausing import for 10 minutes. Then I'm fed up, and shut HM2 down, but, and this is very nice, the import automatically proceeds when restarting HM2. So I can't quit this process. What to do? I'm getting really tired of this.

Final question. What's faster: HM1 import, or import HH's from HMArchive?

[DogNamedBluff]

- HM2 startup often takes 2 mins. Is this normal?
HM2 caches a lot of data and this happens when you launch the program. Although the wait to start is longer by caching the data, we can run extremely fast queries once the program is open. So it should make it worth the wait.


- HM1 DB import is going nowhere. I tried to import 4 days of HH with observed HH. Maybe a part of this was already imported. Took 5 mins to see 0 out of 3.x mio hands imported. This stays like this without changing or getting the impression of any hands being imported. When I press pause, it says pausing import for 10 minutes. Then I'm fed up, and shut HM2 down, but, and this is very nice, the import automatically proceeds when restarting HM2. So I can't quit this process. What to do? I'm getting really tired of this.
It can take some time to import a large amount of hands from 1.0. If you would like to stop the process. Hit Pause then Cancel in the bottom task bar. Otherwise, deleting "MassImportArgs.xml" in the database "Importing" folder will prevent the import from starting when HM2 is opened.



Final question. What's faster: HM1 import, or import HH's from HMArchive?

It is faster to import from the HMArchive.

[Daniel]

No cancel button appeared within 10 mins of waiting after pressing pause button.

[Daniel]

Where is MassImportArgs.xml located? Can't find it.

[udbrky]

Deleting "MassImportArgs.xml" in the database "Importing" folder (AppData\Roaming\HoldemManager\Database\THeNameOfY ourDatabase\Importing

in:


Where is my log file?
7: C:\Users\{username}\AppData\Roaming\HoldemManager\

Showing hidden files: How to show hidden files in Windows 7 (http://www.bleepingcomputer.com/tutorials/tutorial151.html)

XP: c:\documents and settings\{username}\appdata\HoldemManager

Showing hidden files: Microsoft Corporation (http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/win_fcab_show_file_extensions.mspx?mfr=true)