The problem is that you're not playing nice with the CPU. When your program goes into that tight loop reading the lines, the thread scheduler sees you're not waiting on anything from the OS and will continually schedule the thread. What I would do is the following:
- Call SetTimer and tell the API to call back one of your functions. I'd set the elapsed time to something like 750 milliseconds or even a second.
- In your timer function, I'd read three to five lines and insert those lines into the database before ending.
By using the timer, you'll break up the processing and thus ensure you're not hogging the CPU.
This was first published in February 2003