Its just depends on when ever your W-4 was filed and processed. It can take up to a month after you were hired. Or sounds like you where hired or your paperwork was held up during the Govt Shutdown.
If you put in money to be withheld from each check, the less likely you will have to pay at tax time. Depending on your hours and paygrade, you may want to $5 to $20 or so withheld each pay period.
Yes you will eventually pay those missed tax dollars; they will come from somewhere at some point, probably tax time.