Consolidated Statements of Cash Flows - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2015	Dec. 31, 2014	Dec. 31, 2013
Operating activities
Net income (loss)	$ 51,620	$ 56,361	$ (134,136)
Adjustments to reconcile net income (loss) to net cash provided by operating activities:
Net amortization of premium related to mortgage-backed securities	117,654	101,979	165,924
Reclassification of deferred swap net loss	30,715	81,132	25,816
Share-based compensation expense	4,256	3,612	2,594
Net realized (gain) loss on sale of securities	(32,731)	(5,196)	283,012
Net unrealized gain on MSR	(10,711)
Net unrealized (gain) loss on mortgage loans held for investment	1,065	(8)
Purchase of mortgage loans held for sale	(130,362)
Sale of mortgage loans held for sale	117,664
Net loss on derivative instruments	188,416	141,433	68,205
Impairment loss on mortgage-backed securities			8,102
Other	(753)	226	106
Changes in operating assets and liabilities:
Decrease in accrued interest receivable	9,266	882	21,957
Increase in other assets	(19,318)	(2,963)	(2,432)
Increase (decrease) in accrued interest payable	(2,030)	(1,311)	687
Increase in other liabilities	13,480	2,336	572
Net cash provided by operating activities	338,231	378,483	440,407
Investing activities
Purchases of securities	(3,764,409)	(5,126,375)	(10,180,894)
Principal repayments on securities	3,920,850	3,423,756	5,782,709
Sales of securities	2,796,515	1,993,707	11,280,146
Purchases of mortgage loans held for investment	(357,996)	(31,462)
Principal repayments on mortgage loans held for investment	44,105	10
Purchases of mortgage servicing rights	(259,181)
Acquisition of Pingora, net of cash acquired	(21,473)
Net payments on derivative instruments	(87,933)	(53,192)	(67,656)
Purchase of equity investment		(6,000)
Purchase of broker/dealer		(1,349)	0
Net cash provided by investing activities	2,270,478	199,095	6,814,305
Financing activities
Repurchase of common stock	(20,752)	(1,912)	(43,881)
Cash dividends paid	(210,646)	(215,098)	(283,309)
Proceeds from borrowings	157,899,476	151,439,951	226,824,755
Principal repayments on borrowings	(160,145,185)	(151,809,803)	(233,561,501)
Proceeds from collateralized trust borrowing	59,466
Repayments on collateralized trust borrowing	(1,948)
Proceeds from dollar roll financing		241,121	1,784,998
Repayments on dollar roll financing		(592,947)	(1,433,172)
Payment on termination of interest rate swaps and swaptions			(3,342)
Net cash used in financing activities	(2,419,589)	(938,688)	(6,715,452)
Net increase (decrease) in cash and cash equivalents	189,120	(361,110)	539,260
Cash and cash equivalents, beginning of period	627,595	988,705	449,445
Cash and cash equivalents, end of period	816,715	627,595	988,705
Supplemental disclosure of cash flow information
Cash paid for interest	117,767	166,368	197,022
Supplemental schedule of non-cash investing and financing activities
Obligation to brokers for unsettled purchases of securities	12,582	24,750
Receivable for securities sold		5,197	231,214
Dividends accrued not yet paid	47,824	53,014
Redeemable Preferred Stock
Supplemental schedule of non-cash investing and financing activities
Dividends accrued not yet paid	4,628	4,628	4,628
Common Stock
Supplemental schedule of non-cash investing and financing activities
Dividends accrued not yet paid	$ 43,196	$ 48,386	$ 48,301