If the connection is dropping out of the connection table you should see an entry in the logs saying drop because first packet isnt SYN.

Should be quite easy to see.