Well:
1. He immediately signed over a huge asset, the Panama Canal, for nothing at all, after all of the U.S. money made to build it.
2. The Shah of Iran was a huge U.S. ally and moderate voice in the region. When he was challenged by a band of extremists, Carter refused to step in and help, leading to the Islamic Revolution and the Iranian leaders we see now. This also caused the Iran hostage crisis, and Carter was pretty weak in negotiations. He did attempt a military rescue, which went horribly wrong.
3. High unemployment coupled with high inflation and high interest rates. The infamous malaise speech, where he attacked Americans for becoming fat and lazy (he was right, but you don't say that!)
4. Energy crisis, oil crisis.
5. 3 mile Island. And in hindsight, perhaps the worst decision was to essentially end new nuclear plants and the decision not to run breeder reactors, i.e. trying to store the spent fuel when its perfectly good fuel itself.
6. Upset the farmers by not supporting grain exports to Russia.
7. Gave tons of money to Nicaraguan revolutionists, who used it to set up a totalitarian regime not friendly to the U.S.
8. His brother recieved a huge kickback from Libya to facilitate oil sales. Libya was not exactly well liked at the time.
Not saying its all his fault, but a lot of bad stuff happened while he was in office.