His actions have shown that he has quite a few reasons for not wanting you to file for divorce, none of which have anything to do with his love for you or his commitment to the marriage. You've identified a few reasons (losing face with his daughters, public shame) but here's a few potential (but likely) other reasons:
1. Check with a lawyer but if your state recognizes adultery as grounds for divorce, then there could be legal/financial consequences for your husband. You might also be able to list is OW as his adultery partner in the divorce papers, which would expose her.
2. He doesn't want his OW's husband to find out about their affair.
3. He doesn't have his ducks in a row yet (legally or financially) so he wants to you to hold off on filing for divorce until he can blindside you with filing at a time that is strategically advantageous to him.
Given the reasons above, the fact that he refused to give up his OW, and that he's currently is using marital funds to conduct his affair, you really don't have a choice other than to retain a lawyer and start the process yourself. You have nothing to work with.
If you're anxious about starting the process, let me share the following cautionary tale:
When my aunt was in the process of divorcing her a-hole husband, she started off with the upperhand. Her lawyer was outmaneuvering her ex's, and the judge had been ruling consistently in her favor. At the 11th hour, her ex told her that he wanted to try for reconciliation. She stopped the divorce. Her lawyer advised her against it but she didn't listen (after all, the lawyer doesn't get a big pay day if she stays married).
For a year, my aunt tried to work things out with her a-hold husband and he reneged on every promise he made to get her to stop the divorce. She decided to start the process up again, but a lot had happened in that 1 year, namely the 2008 financial crash, which depleted the value of their shared assets and decreased his bonus-based income, and the original judge who had been assigned to her case retired and was replaced by a new judge who was much less sympathetic toward her. Plus, her lawyer didn't feel motivated to fight with the same zeal as she did previously.
Long story short, my aunt got screwed... not only out of money and assets, but out of time and dignity. Worse, on top of the pain she suffered from the divorce process, she had the added trauma of trying to recover from the absolute mind-fuck that is false reconciliation.
My advice in summary: Do not get caught flat-footed. Start the process now. If your husband begs and pleads for you to stay, tell him to put his mouth where his money is and give you a generous settlement that goes above and beyond what you would be entitled under the law. Then, when the divorce is finalized, you'll consider dating him.
Trust me, you'll have a perfectly clear picture of his intentions after making an offer like that.
[This message edited by BluerThanBlue at 5:47 PM, Tuesday, September 10th]