Does the 800x600 resolution scale it out over the screen?
If so, just make is so that the resolution isn't stretched helps a tone in getting more FPS.
But you'll have to do with black borders around the actual screen.
The actual ripping starts at around 175+ FPS if your FPS is inconsistent.
When you are on a constant 180fps you'll have a constant jitter.