Do it! I haven't taken my boys mainly because I'm "working" but to be able to see the cars up close and talk with the folks in the pits is awesome. Both Monteray and Portland had great areas to either sit in grandstands or lean on a fence. Bring a back pack with fluids/snacks and a camera and you won't be sorry.
Dan