I drove to Fernacre last year. Just pulled off the track and parked then walked across.
Several landies passed me but none stopped to investigate. I was nervous driving up to the cross, but once there, I just carried on. Actually, looking a the Cornwall Mapping service for public rights of way http://mapping.cornwall.gov.uk/website/prow/ it seems that it's all access land anyway, so certainly no problems walking there if you're fit enough. A nicer walk might be from the north of Rough Tor. Park in the car park, then over/round Rough Tor itself. Lots of archaeology up that way to see!