Commercial Mortgage Banking - Narrative (Details) - USD ($)	3 Months Ended		9 Months Ended
	Sep. 30, 2016	Sep. 30, 2015	Sep. 30, 2016	Sep. 30, 2015	Dec. 31, 2015	Dec. 31, 2014
Mortgage Banking [Abstract]
Prepayment costs	$ 0	$ 4,900,000	$ 6,500,000	$ 20,900,000
Custodial escrow accounts	2,400,000		2,400,000		$ 421,500,000
Uninsured balance relation to escrow accounts	1,600,000		1,600,000
Fair value of servicing rights	273,000		273,000		41,800,000
Carrying value of loan servicing rights	$ 268,000	$ 29,594,000	268,000	$ 29,594,000	28,168,000	$ 30,140,000
Servicing rights are amortized using straight-line method	10 years
OMHHF
Mortgage Banking [Abstract]
Carrying value of loan servicing rights	$ 268,000		$ 268,000		$ 28,168,000