My coworker at NCTM, Beth Skipper, is the editor of Teaching Children Mathematics, the journal for elementary school math teachers. She is very knowledgeable about elementary school mathematics, and she is a plethora of information about books for children that involve math. She recently started a blog at http://bethsbookshelf.blogspot.com that gives a tip every day for using various books to help teach mathematical concepts. I highly recommend it. Check it out!
My favorite is "Hippos Go Berserk" by Sandra Boynton. First there's one hippo, then two more, then three more, all the way up to nine more... then after the party, nine leave, then eight leave, etc. At the end, it says, "One hippo, alone once more, misses the other forty-four." This is a nice and not-too-serious display of the triangular numbers. I wouldn't reall call this a math book, though -- it's just simple counting, really. But after we read it, we talk about some of the bigger ideas, such as why the hippo misses forty-four others, and we figure out that 2 + 3 + ... + 9 = 44.
As for true math books, we haven't used too many. I've generally been frustrated by them -- the books at their reading level focus on concepts that are below their math level. Generally, I take other books we have and try to draw out the math that's in them. Or I'll create a story on my own that has some math in it. (Maybe there's a market for them?)
On most nights after dinner, Alex will now ask, "Can we do some math before we go to bed?" Partly, this makes me think I'm doing a good job as a parent; but mostly, it makes me realize that my kid is smart enough to know effective stall tactics to prevent him from having to go to bed before he wants to.