Taxes - Components of Net Deferred Tax Assets (Details) - USD ($) $ in Thousands	Dec. 31, 2016	Dec. 31, 2015
Deferred Tax Assets
Net operating loss carryforward – state	$ 89,350	$ 95,972
Net capital loss carryforward – state	15,346	22,603
Net operating loss carryforward – federal	9,537	32,929
Net capital loss carryforward – federal	2,283	7,971
Real estate assets	5,601	5,144
Interest rate agreements	0	1,472
Allowances and accruals	3,059	3,458
Other	2,192	513
Total Deferred Tax Assets	127,368	170,062
Deferred Tax Liabilities
Mortgage Servicing Rights	(22,531)	(50,630)
Interest rate agreements	(2,167)	0
Tax effect of unrealized gains – OCI	(1,636)	(2,638)
Total Deferred Tax Liabilities	(26,334)	(53,268)
Valuation allowance	(101,932)	(116,794)
Total Deferred Tax Asset (Liability), net of Valuation Allowance	$ (898)	$ 0