Linking ordinarily would make you a good neighbor and a good citizen of the internet. However, just because someone links to you does not obligate you to link back to them. Sometimes you do get judged on the quality of the friends you keep. Google calls spamming sites bad neighborhoods, and linking to them could lower your PageRank.
Link exchanging programs, paid link placement, and other schemes to manipulate PageRank are far worse sins.
You may get away with it for a while, but eventually, Google will catch onto the scheme, and your search results will drop like an anchor. This is essentially what happened in the JC Penny's incident. The SEO firm they hired (and subsequently fired) created an artificial web of links on unrelated websites.
