Custom monsters, animations, possible?

It is possible, the only thing you need to do is find existing 3d monsters / npc models and convert into bmd ( using some import/export tools ), or you can create your own models with some 3d software.

You need to know that when you attempt to create a monster.bmd, you need to export at least basic animations in the right order ( stand01, stand02, walk, attack01, attack02, ReceiveHit, Die).
Finally you can add new animations if u want... To make new animations work you need to crack client to play animations requested by server.

Here is an example.