I 'visited' this site two years ago when visiting Lydney Park Gardens (very nice)
The gardens are not open all year round so you see to check for opening details before planning a visit.
Parking is on a grass field at the bottom of a small valley.
Directly in front of you is the hill on which the remains of the Hillfort sit. A path to the left takes you up the steep incline to the remains of the Roman Temple which lies within the Hillfort. I know the Romans are not everyone's cup of tea who use this site but the ruins are worth checking out – temple / bath house etc.
Unfortunately I wasn't aware of the Hillfort when I visited so didn't make a special effort to look for clues to its remains. The centre of the Hillfort is pretty flat and has been largely kept clear of trees etc to protect the Roman remains.
There is a pretty good museum in Lydney House although I don't remember seeing any prehistoric items.
"On the summit of Camp Hill in Lydney Park, one mile south-west of the town, there are the remains of a fine Roman villa and temple, protected by earthworks running across the promontory and along its eastern side. On the west side there are no earthworks, the escarpment of the hill being very steep. On the north side, where the ridge of the hill is prolonged, there is a deep ditch outside the mound, and at a little distance from this is a second mound. The area thus defended measures about 280 yards by 120 yards, the buildings occupying the south-eastern portion. The situation is admirably selected for defence, the steep slopes of the hill forming a difficult approach on three sides, and water being easily obtained from a fine stream running on the north side. An approach road is still to be traced running up to the camp, and corresponding with the upper end of this is a gateway in the outer wall of the Roman villa. This road has been considerably lowered in comparatively recent times, but part of the original trackway can be seen just before entering the camp. This was doubtless a very important position during the Roman occupation of this country. The series of coins found extend from Augustus, who died A.D. 14, to Arcadius, who died A.D. 408."
From "Archaeological Handbook of the County of Gloucester" - George Witts (1883)