We've had good luck with the refurbished system we bought from Dell Financial Services
on eBay. Models tend to be a bit older than the ones at the Dell Outlet store, which also runs some good deals (as long as a shipping discount is available).
I went with a GX240 desktop because I already had accessories and compatible upgrades (RAM, DVD-R drive, etc.) from my previous system. It's a P4/1.5GHz that is used for tasks similar to the ones you list, and it's more than enough. It came in at less than $150 shipped over a year ago and included a brand new keyboard and optical wheel mouse.