Black - Top Row
In this sequence I have started at the camera suggested optimal exposure and then progressively underexposed by 1/2 stop. Even at 2 stops underexposed the black is still reading at an RGB of (50,50,50), so still a very dark grey. I suspect that the surface of the felt is slightly reflective and so would need at least another stop of under exposure to get a pure black. I noticed this when taking shots around town of black objects, most are very shiny surfaces and so although the hue is technically black so much light is reflected that it is nearly impossible to get an accurate black, without using the lens cap.
White - Middle Row
Here I have started again at a nominal ideal exposure, but opened up by 1.2 a stop in each successive image. Even at two stops over the white is still quite grey (227,227,227).
Grey - Bottom Row
The final sequence is bracketed around the ideal exposure, so from -1 to 1 in half stop increments. In this case the middle image at the "ideal" exposure is closest to where it should be at (120,120,120). The grey I used was slightly paler than my reference grey card, so the proper image is probably the next one up.
What is also clear from the 3 separate images is that the camera has taken each of the different shades and rendered them to a neutral grey, all of the "ideal" shades are within 15/255 apart on the RGB scale.
I also shot image of all 3 shades together for comparison, this time setting an accurate exposure with a grey card. It is quite striking how different the shades of grey are, and yet the camera happily decided they were all the same.
On this combined image the black is around (2,2,2), the grey is around (145,145,145) so lighter than neutral grey and the white is (230,230,230), so really a very light grey. Comparing the white felt with a sheet of paper it is clearly on the gray side of white.
I was careful in this set of images to white balance the camera using the grey card before shooting as I wanted to reproduce the grays accurately, thus there is no colour caste in any of these images. In reality most shots I have taken of grey scenes contain a caste driven by the colour of the ambient light.
So the lesson is, careful trusting a cameras light meter where there is limited colour variation, use a handheld meter or grey card and always shoot RAW!