Color perception and SRM measurement don't necessarily correlate. The mechanics of wort viscosity and Maillard reaction have several variables that are not going to be picked up by SRM in a meaningful way. Someone's observation of beer color depends on turbidity, depth of the sample, light source and how close you hold it to your eye.
SRM measures one light wavelength in the deep blue spectrum (430 nanometers). A true SRM reading is 1 cm thick, free of turbidity. The number is derived from the log ratio of the light entering vs. the light exiting the sample, multiplied by 12.7. Since it uses just one wavelength, the SRM scale is not very good at detecting the difference between light orange and deep burnished gold or other colors based on subtle red shifts.
Bob Hansen at Briess spent a lot of time on the subject and created this poster for the MBAA, http://www.brewingwithbriess.com/Assets/Presentations/Briess_2010MBAA_BeerColorPoster_.ppt
To anyone without a lab, the use of SRM for color observation is a bit arbitrary and possibly useless. Most observable beer color shifts are in the red area of the spectrum. This is more typical of the wort darkening you're describing. There is a newer formula for SRM that takes more coefficients into account and may make the numbers more likely to change according to human perception, but it isn't getting much traction.
The ASBC has approved a method that uses 81 different wavelengths, which places SRM into a scale that recognizes more than just the loss of 430 nanometers. If adopted by the industry, it would change the way numbers represent the color of beer.