I've just been glancing over this thread, but here is my thought.
I'm assuming your input voltage is 220-240VAC. Rectified and filtered this would be around 340VDC, give or take a little bit. You are running a half-bridge topology, so the voltage being switched across the primary of your transformer will only be half of your input bus voltage, so about 170VDC. Now, given the details of your transformer, your turns ratio is 3.54:1, the maximum peak voltage coming from your transformer absolutely cannot be higher than 48 or so volts.
Now, having said that, I can only think of two possibilities of what is going on here.
1.) The primaries of your transformer accidentally got connected in parallel instead of in series. This would cause the output voltage to be doubled, but also would most likely cause the transformer core to saturate (or get really close to it) which would cause the output voltage to drop significantly under load and cause the transformer itself to get quite hot. This would also cause heating in the mosfets and could also cause them to fail.
2.) The transformer is being driven as a flyback transformer. For example, if one of your mosfets isn't switching properly. These IR2153 chips tend to be quite fragile and I've seen many examples of the high-side driver just completely not working at all. If you've ever had a mosfet fail on this circuit, you should replace the IR2153 at the same time you replace the mosfets, because it is most likely failed also. In this case, if only your low-side mosfet is working, this could cause your transformer to saturate, run very hot, and due to the inductive flyback effect, cause a very high output voltage under no load conditions with a severe drop in voltage with any significant load applied.
Your transformer only has a single secondary winding, so I'm not at all worried about the connections on that side. If the output diodes were reversed, it would either dead short the output (quickly killing the output diodes) or cause no output voltage at all.
Without a scope you're just shooting off in the dark here, but those are my two ideas. Good luck and I like the professional board design.
- Brad