I go a Karen's 2-3 times per year, and always on a Saturday morning when they are very slammed. Yes, sometimes the food can take a long time, up to 20 minutes, and they don't always come by to offer refills, but are always happy to do so at the counter. Haven't had a problem with tables not cleared, but sometimes we've had to wait a few mins for customers to leave before a table becomes free. Food has always been good.
In general, I've found that bakery-restaurant places like Karen's tend to have slower service. The counter staff are always juggling taking orders, making coffee drinks, getting goods from the bakery cases, handling phone orders, retrieving take out pickup.