I had a similar problem with a street rod. It was so sporadic it drove us nuts. LONG story short the starter wasnt engaging properly with the flex plate. The starter wasnt shimed properly and the selenoid would throw the shaft and get lodged in the teeth making NO noise or sign of trying to crank. Sometimes it would engage and sometimes it wouldnt.
Try cranking yours when COLD. Do it several times over and over and see if it happens when the engine is cold. If it does you can rule out the hot/cold idea. I know this doesnt explain the low voltage issue, but thought I'd make a different suggestion.
Another idea is like Trey suggested. Timing could be off that when it is hot it wont crank.
Either way... I feel for you. Electrical gremlins are a PITA to diagnose.
Good luck..... be sure to let us know what is was when you find out.