First of all, do you have enough money to pay points at closing? For some buyers, there just aren't enough funds to pay points at the time they buy their home. However, it is important to remember the long term savings. For some, a parent or family member may be willing to gift you the money.
Secondly, how long do you plan on keeping your home? If it will take you 32 months to realize savings from paying points, and you know you will be keeping your home at least until your youngest child starts college (7 years, or 84 months from now) then it makes sense to pay points since there would be 52 months of savings.
Lastly, keep in mind that the decision to pay points is not just about how long you plan on keeping your house. It has to do with how long you plan on keeping your loan. If you know you will be doing a major remodel in one year and then refinancing or obtaining a construction loan, then it may not make sense to pay points.
Your loan officer can show you in real life numbers if it makes sense for you to pay points, offering several different scenarios for you to choose from.