Well, I guess you could do such an awful job at it a few times that eventually you won't be roped into it again. But then you might find yourself barely scheduled for anything so that's not a great option.
Is your unload chronically understaffed? Someone on vacation or on LOA? If this is a short-term problem, I'd suck it up and just get through it. If this is a long-term problem, I'd talk to my TL about it (at a different time other than when I'm needed to help with unload). But I probably wouldn't whine about "not what I was hired in for" because, as others have pointed out with the "business needs" part, that's not really true. Is someone else pushing the Babies freight and doing a bad job of it? Focus on what you bring to the job.