Your ISP,
Timer Warner, Adelpia, Cox, AOL, one of the Bells, whomever can intercept the packets of data that is sent over the connection that they provide to you. If these packets are not encrypted, then yes they could see what you are doing.
They, for the most part don't give a flying fuck unless you are becoming a burden on their network (heavy downloading/uploading).
Now, are they gonna turn you over to the local authorities for downloading copyrighted material? No. It's not their copyright to enforce. But it is conceivable, and I have read reports where the ISP has taken it upon them selfs to issue warnings, listing downloaded material. These are usually an attempt to get you to use less bandwidth, more than anything else. Bandwidth is expensive to create and it is in their best interest if they can get you to use as little as possible.
What you are thinking of is the MPAA/RIAA/ and other copyright holders in general. No, they can not see what you download like on P2P networks. No one has ever been sued for downloading off Usenet, their is no way for these outside parties to get into the loop.
SSL encryption is only necessary if you have had problems with your ISP in the past. SSL is to hide what you are doing from them, for whatever reason. Whither it is because they are using some type of traffic shaping tools, or you just don't trust them to keep their noses out of your business. In all reality, most don't care.