The game automatically detects which of your Facebook friends have also connected to Family Guy and creates your friends list. In order to remove a specific friend, you'll need to unfriend them on Facebook.