Taxes - Components of Net Deferred Tax Assets (Details) - USD ($) $ in Thousands	Dec. 31, 2017	Dec. 31, 2016
Deferred Tax Assets
Net operating loss carryforward – state	$ 108,085	$ 89,350
Net capital loss carryforward – state	0	15,346
Net operating loss carryforward – federal	0	9,537
Net capital loss carryforward – federal	535	2,283
Real estate assets	0	5,601
Interest rate agreements	1,380	0
Allowances and accruals	2,044	3,059
Other	1,844	2,192
Total Deferred Tax Assets	113,888	127,368
Deferred Tax Liabilities
Real estate assets	(562)	0
Mortgage Servicing Rights	(20,540)	(22,531)
Interest rate agreements	0	(2,167)
Tax effect of unrealized gains – OCI	(1,166)	(1,636)
Total Deferred Tax Liabilities	(22,268)	(26,334)
Valuation allowance	(103,384)	(101,932)
Total Deferred Tax Asset (Liability), net of Valuation Allowance	$ (11,764)	$ (898)