Hi,
I don't think anyone's likely to think you're an unfit mother -- in my experience (as a dad), the services did everything they could to keep everyone together.
If you're awaiting a referral, I wouldn't try and be too worried. If they were worried, it would be a pretty instant referral (i.e. within 24hrs), so, from what you say, I reckon it's precautionary.
Rest assured there are plenty of people going through it. You are not alone. Sorry no-one else has replied yet, but there is support available, particularly in the puerperal psychosis forum where we have a couple of fairly regular contributors. You might want to try posting there?
I hope all goes well for you and wish you the best,
D.