Consolidated Statements Of Income - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2016	Dec. 31, 2015	Dec. 31, 2014
Interest Income
Residential loans	$ 137,804	$ 114,715	$ 68,949
Commercial loans	30,496	46,933	47,567
Real estate securities	76,873	97,448	125,482
Other interest income	1,182	336	72
Total interest income	246,355	259,432	242,070
Interest Expense
Short-term debt	(22,287)	(30,572)	(25,990)
Asset-backed securities issued	(14,735)	(21,469)	(31,227)
Long-term debt	(51,506)	(43,842)	(30,246)
Total interest expense	(88,528)	(95,883)	(87,463)
Net Interest Income	157,827	163,549	154,607
Reversal of (provision for) loan losses	7,102	355	(961)
Net Interest Income after Provision	164,929	163,904	153,646
Non-interest Income
Mortgage banking activities, net	38,691	10,972	34,994
Mortgage servicing rights income (loss), net	14,353	(3,922)	(4,261)
Investment fair value changes, net	(28,574)	(21,357)	(10,202)
Other income	6,338	3,192	1,781
Realized gains, net	28,009	36,369	15,478
Total non-interest income, net	58,817	25,254	37,790
Operating expenses	(88,786)	(97,416)	(90,123)
Net Income before Provision for Income Taxes	134,960	91,742	101,313
(Provision for) benefit from income taxes	(3,708)	10,346	(744)
Net Income	$ 131,252	$ 102,088	$ 100,569
Basic earnings per common share (in dollars per share)	$ 1.66	$ 1.20	$ 1.18
Diluted earnings per common share (in dollars per share)	1.54	1.18	1.15
Regular dividends declared per common share (in dollars per share)	$ 1.12	$ 1.12	$ 1.12
Basic weighted average shares outstanding (in shares)	76,747,047	82,945,103	82,837,369
Diluted weighted average shares outstanding (in shares)	97,909,090	84,518,395	85,098,579