PDA

View Full Version : HEM Will Not Import (All Errors)



icemonkey9
01-13-2010, 12:14 AM
I have tried importing some hand histories that were exported from my other machine. Both machines are running 1.10 Beta 6. The machine I am getting errors on is a Windows XP system running Postgresql 8.3 . No matter what I try, whether it be purchased hand histories or exported hands I played myself on my other machines - EVERY hand comes in with an import error.

The issue:

When I import "from a folder" it will not import any hands and say import errors and give me the number of hands I am importing.

When I try to do this "import from a file" it does the same thing.

I tried even copying the hands into the hand history folder for pokerstars and the same result happens, all hands I try to import come in as errors.

I am totally out of ideas.

netsrak
01-13-2010, 05:36 AM
Please try with a new database

icemonkey9
01-15-2010, 02:02 PM
Yes, a new database (actually going back to the old one) worked. Not sure what else there is to say.

KweezyD
02-15-2010, 01:21 PM
If the powers that be don't mind...I'd like to re-open this thread b/e I have experienced all of the above (import errors as described by OP; and a new database as a fix) AND then some.

HEM began importing correctly once I created a new database. Alas, this was only temporary, as the "every file to be imported experiences an import error" problem began again in earnest shortly after creating the new database and importing maybe 200K hands or so.

I didn't always have this problem. I've been using HEM for 6 months now, and have had these problems only for the last 2.

morny
02-15-2010, 03:06 PM
The 2 most common causes of this problems are 1) powering off your computer or having your computer lose power while the database is doing something and 2) an old drive

with bad sectors on it. Basically what happens is a small portion of a file gets slightly corrupt and then when you reboot and windows goes into that checkdsk procedure it

finds the bad section and, thinking it is doing a good thing, removes it. Postgres then loads the table and a portion of it is gone so it doesn’t like that and blocks

access to the entire table. Normally when this happens it is on a completely useless file that you'll never even notice is now gone and quite often it will happen to an

index in the DB which can easily be rebuilt but in your case it happened to one or more of the tables.

So, to protect against this

1) keep you hands histories in case you do need to reimport
2) use a power bar and avoid hard shutdowns (when you flick the power switch on the pc)
3) if your drive is old consider replacing it with a new one. They are cheap and much faster now than even a few years ago.

FWIW ive been using HM for probably 2 years, as part of internal testing we try out internal betas which would have more bugs than the versions we release, i regularly import hands people send that they cant get to import, i datamine 24 hours on a networked PC and i play poker regularly using HM and ive only gotten 1 corruption when my PC lost power during an import so it is extremely stable so if your having issues its most likely something on your PC

hero will drown
02-15-2010, 03:56 PM
The 2 most common causes of this problems are 1) powering off your computer or having your computer lose power while the database is doing something and 2) an old drive

with bad sectors on it. Basically what happens is a small portion of a file gets slightly corrupt and then when you reboot and windows goes into that checkdsk procedure it

finds the bad section and, thinking it is doing a good thing, removes it. Postgres then loads the table and a portion of it is gone so it doesn’t like that and blocks

access to the entire table. Normally when this happens it is on a completely useless file that you'll never even notice is now gone and quite often it will happen to an

index in the DB which can easily be rebuilt but in your case it happened to one or more of the tables.

So, to protect against this

1) keep you hands histories in case you do need to reimport
2) use a power bar and avoid hard shutdowns (when you flick the power switch on the pc)
3) if your drive is old consider replacing it with a new one. They are cheap and much faster now than even a few years ago.

FWIW ive been using HM for probably 2 years, as part of internal testing we try out internal betas which would have more bugs than the versions we release, i regularly import hands people send that they cant get to import, i datamine 24 hours on a networked PC and i play poker regularly using HM and ive only gotten 1 corruption when my PC lost power during an import so it is extremely stable so if your having issues its most likely something on your PC

This is exactly what has happened to me, how do I get my HEM back to normal?

netsrak
02-15-2010, 04:14 PM
You have problems with your database or why are you asking?
What error message do you get?

KweezyD
02-15-2010, 04:55 PM
The 2 most common causes of this problems are 1) powering off your computer or having your computer lose power while the database is doing something and 2) an old drive with bad sectors on it.

Thanks for your entire explanation. Sounds a lot like it's (#2). But my Dell Inspiron PC is <3 years old. Must just be a bad hard drive.

KweezyD
02-15-2010, 05:03 PM
You have problems with your database or why are you asking?
What error message do you get?

No problems with the database. Problems only with importing batch HH's (purchased from Poker Table Ratings...say up to 2 million HHs per batch).

It doesn't happen ALL the time...only about 90% of the time. During the import process, each file being imported fails to import and ends up in error. For example, here's what my Import tab will typically read as. This is from the IMPORT PROGRESS: box....

Files Imported: 100 of 100
Hands Imported: 0 Hands
Duplicate Hands: 0
Import Errors: 100,000

KweezyD
02-15-2010, 05:46 PM
The 2 most common causes of this problems are 1) powering off your computer or having your computer lose power while the database is doing something and 2) an old drive with bad sectors on it.

(1) Come to think of it, could the porn gods have sent my computer some kind of trojan / virus that may be causing this?
(2) Also, what about this...I'm not plugged into a battery backup, only a surge suppressor. And I have no idea how consistent the electrical currents are in my older apartment. Could the Postgress database be corrupted easily by problems with a clean/consistent power supply--be it as severe as a brown-out, or something smaller, like an unsteady or weak electrical current in my
building?

netsrak
02-16-2010, 04:38 AM
If you want us to check the hands please zip and mail them with a link to this thread to support@holdemmanager.net

rickosu
02-16-2010, 05:24 AM
No problems with the database. Problems only with importing batch HH's (purchased from Poker Table Ratings...say up to 2 million HHs per batch).

It doesn't happen ALL the time...only about 90% of the time. During the import process, each file being imported fails to import and ends up in error. For example, here's what my Import tab will typically read as. This is from the IMPORT PROGRESS: box....

Files Imported: 100 of 100
Hands Imported: 0 Hands
Duplicate Hands: 0
Import Errors: 100,000

I have this EXACT same problem with PTR hand histories. Any fix for this?

rickosu
02-16-2010, 05:27 AM
If you want us to check the hands please zip and mail them with a link to this thread to support@holdemmanager.net

Email sent! I appreciate your help.

rickosu
02-16-2010, 05:35 AM
Email sent! I appreciate your help.

I just tried the suggestion of making a new database and tried to import the files to it and now the hands will import.

It's weird because my old database will still import live hands while I'm playing, but it won't import these bulk hand histories that I purchased. I looked at the format of the files and compared them and they appear to be exactly the same. This all started happening after I did a dump/restore of my postgres and vacuum/reindexed it.

Any idea what could be causing this?

netsrak
02-16-2010, 11:00 AM
I guess something went wrong during your dump/restore.
How did you create the new database. The correct way is via pgadminII with the same coding as the original database.

KweezyD
02-16-2010, 03:12 PM
(1) I just tried the suggestion of making a new database and tried to import the files to it and now the hands will import.

Ditto for me.



(2) It's weird because my old database will still import live hands while I'm playing, but it won't import these bulk hand histories that I purchased.

Ditto for me.


This all started happening after I did a vacuum.

Ditto for me.

Support: With what probability, would you say, the issue described above in this email is being caused somehow by a database vacuum?

Here's my vacuum experience:
(1) I created a new database
(2) bought a 2 million+ batch of HH's and began to import them.
(3) 8 hours later, the import was still going, and I stopped the import, thinking I should vacuum the database in order to speed the import process up - since it had gone from, say, a 70 hands/second import rate down to 5 hands/second rate.
(4) I don't really know what "vacuuming the database" actually means...I've been assuming it means something similar to defragmenting one's computer hard disk to regain performance.
(5) I stopped the import, then ran a vacuum. And ONLY a vacuum.
(6) I'm not sure exactly when, but at some point during the subsequent import, the error messages began again.
(7) The error messages continue for, I'd guess, 90% of all my subsequent attempted HH imports, with the constant and never-changing exception of live HH's while I play poker, which always import like champs.

KweezyD
02-16-2010, 03:20 PM
How did you create the new database. The correct way is via pgadminII with the same coding as the original database.

This is how I created the database in question:
(1) open HEM
(2) Options---->Database Management
(3) Database Control Panel------->Connect
(4) Database Control Panel------->Create New Database

morny
02-16-2010, 04:37 PM
When you created a new database did you use UTF or ASCII, whichever one you used last time try the other one and see if that makes a difference.

if there still is problems, zip up the hands and email/upload them and send them to support@holdemmanager and also link to this thread

KweezyD
02-17-2010, 06:11 PM
When you created a new database did you use UTF or ASCII, whichever one you used last time try the other one and see if that makes a difference.

Thanks. I did this, and my test results were nearly identical between the two database types. Here are the results, from the Holdem Manager Import tab, "Import Progress:" box PROGRESS...

Test 1 results (ASCII encoded database):
Files Imported: 2234 of 2234
Hands Imported: 335,013
Duplicate Hands: 0
Import Errors: 1,004,747

Test 2 results (UTF encoded database):
Files Imported: 2234 of 2234
Hands Imported: 120,616
Duplicate Hands: 0
Import Errors: 1,291,144





if there still is problems, zip up the hands and email/upload them and send them to support@holdemmanager and also link to this thread

Email has been sent. Thanks.


EDIT: For the record, I created 2 new databases, and then ran 2 batch import tests. All hands were imported without interruption: i.e. no vacuum was done to either database at any time.

payday
02-21-2010, 03:47 PM
Unfortunately I now have the same problem where I get all import errors with a large number of mined hhs but live hhs are imported fine most of the time.

I do get the same failure messages as in this post / thread:

http://forums.holdemmanager.com/releases/11641-1-09-beta-43-now-official-release-180.html#post85594

http://forums.holdemmanager.com/manager-bugs/2574-importing-error-2.html

Am I right that a dump/restore won't help and my only option is to reimport all the hhs in a new database?

Should I also do a complete reinstall of postgres?

morny
02-21-2010, 04:23 PM
Its possible some of the mined hands are corrupt, try creating a new database and importing a few of them to see, if they import fine into the new database youll need to export your hands from the old database into the new ones as restoring wont fix the issue.

FAQ - Hold'em Manager Poker Tracking Software :: How to Import & Export Hands (http://faq.holdemmanager.com/questions/98/How+to+Import+%26+Export+Hands)

phillitup
02-23-2010, 04:48 AM
I am encountering these same problems, 95% of bulk HH won't import. Funny thing is, when I break up a 1000 hand file into 6 or so smaller files thru notepad by cut and paste they import just fine. However if I cut and paste the whole file, it won't import.
I originally figured there must be one or more hands in a file that were corrupt. So I started chopping the HH into smaller bits trying to locate the problem. There was no problem with the HH.
Could it be that Postgres or HM is now having problems with large files?

payday
02-23-2010, 08:49 AM
I can confirm what phillitup wrote, the HHs are fine and they import without a problem in new created database, but I do get only import errors with big files when importing them in my old, very big database. Small files working fine though.

fozzy71
02-23-2010, 10:17 AM
I can confirm what phillitup wrote, the HHs are fine and they import without a problem in new created database, but I do get only import errors with big files when importing them in my old, very big database.

The problem is with your old DB, and not the files, if they import to a new DB without issue.

phillitup
02-24-2010, 04:08 AM
Having had the same issues as stated above with large HH files, I ended up rebooting, opening up HM and going straight to import without running any reports, etc. All files imported perfectly, even ones that were rejected multiple times before.
In my previous attempts, I had run multiple reports in both HM and LeakBuster before trying to import these large files. My computer had also been on and running various apps. Although creating a new DB helped, it was still rejecting about 80% of the large files.
So a simple reboot was what was required for me. Dunno, could be Vista. Prolly is.

Macka
07-20-2010, 06:25 AM
I have the same issue, which is very annoying since I regularly buy handhistories. The issue started to show only after last update. Single reboot (as mentioned in phillitup post) did not work for me. I would like to keep the old database if possible, as I need the handhistories there and it would take a week to import them into new one.

Thanks for all the help.
Met.

I use latest version of HM and windows XP.

phillitup
07-20-2010, 06:55 AM
Macka, although rebooting worked for me initially, the same problems kept reapearing with large files.
I did find something that has worked without fail, and that is to place the large files I want to import on a separate hard drive. Anytime I back up my DB the bu file goes to the second hard drive also.

Hope tis helps you.

netsrak
07-20-2010, 06:55 AM
Please try a vacuum/full via pgadminIII
and after successfull completion a reindex via pgadminIII