This happens to me fairly consistently. My ETL of HR says they will speak to me about it and the conveniently forget to, and then get back to me after I complain to various other people around their position.
You can say no. They will use it against you though. I hate that they won't even try...