Tan colored pool walls

[jpcgolf]

My gunite pool walls are covered with a tan colored substance. When I rub or brush a spot, the gunite goes back to it's normal "clean" color. What is this substance and what do I do to get rid of it? Thanks