Unless you have found a way to forgive him, I don’t see any reason you should attend his funeral. Going to a funeral means providing support and concern to members of the family and making a statement about your sorrow at the loss. Unless you can honestly say you share those feelings, don’t attend. On the other hand, don’t make a big deal about not going. You don’t need to tell your story or explain yourself.
Sadly, many people attend funerals to be seen there. Again, you have not lost a friend, rather a tormentor. But you must leave it alone. If someone asks you why you didn’t go, you can say something like, “I’m sorry for his family,” or “He will be missed by his friends.” That should be it, though. Otherwise, keep your feelings to yourself.