RatRater

IMAE, Integrated Malware Analysis Environment

Try RatRater 2. It has a cleaner UI, tells you if a mod is definitely a rat or safe, and flags on stuff RatRater doesn't.
Note: RatRater 2 does not have all the flags from the original RatRater. However it tends to perform better.

Open

A application that looks for known patterns on 1.8.9 mods to detect if they are malicious.
It also has built-in tools to discover more.