I'm in CA as well. I came to Target as a seasonal TM. As soon as I was made permanent, I stopped wearing khaki. All I wear is black pants. There have been a few isolated incidents where temporary TLs would try and enforce khaki only, but nothing ever stuck.
Yes, unfortunately, "playing the game" and thus doing as your told is probably the safest thing to do. But I say fuck that! Meet them in the middle if you must. If you don't wear khaki, then wear some nice slacks, or some other professional looking pants.