Several of the Target stores near me won't take coupons at the self-serve lanes. As was mentioned, each store could have a policy that is slightly different. More than 4 of the same kind is usually not allowed, but some coupons have their own restrictions. I believe the registers automatically reduce the coupon value if the coupon is for more than the item cost.
As for the total number of coupons, I would think there is no limit, but no store wants to tie up a register if you have hundreds, especially this time of year. I think if you talked with a manager, told them your intentions and asked when the best time would be to come in, they would love it. They make money on the sale and the coupons are refunded back to them.