Fair Value of Financial Instruments Recorded basis and Fair Value (Details) (USD $) In Thousands, unless otherwise specified	Dec. 31, 2012	Dec. 31, 2011
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Financial Instruments, Owned, Mortgages, Mortgage-backed and Asset-backed Securities, at Fair Value	$ 611,322	$ 421,096
Securitized mortgage loans, net	70,823	113,703
Derivative Liabilities	42,537	27,997
Carrying (Reported) Amount, Fair Value Disclosure [Member] | Agency MBS [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Financial Instruments, Owned, Mortgages, Mortgage-backed and Asset-backed Securities, at Fair Value	3,492,659	1,965,159
Carrying (Reported) Amount, Fair Value Disclosure [Member] | Mortgage-backed Securities, Issued by Private Enterprises [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Financial Instruments, Owned, Mortgages, Mortgage-backed and Asset-backed Securities, at Fair Value	611,322	421,096
Carrying (Reported) Amount, Fair Value Disclosure [Member] | Financing Receivable [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securitized mortgage loans, net	70,823	113,703
Carrying (Reported) Amount, Fair Value Disclosure [Member] | Other Investments [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Investments, Fair Value Disclosure	858	1,018
Estimate of Fair Value, Fair Value Disclosure [Member] | Agency MBS [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Financial Instruments, Owned, Mortgages, Mortgage-backed and Asset-backed Securities, at Fair Value	3,492,659	1,965,159
Estimate of Fair Value, Fair Value Disclosure [Member] | Mortgage-backed Securities, Issued by Private Enterprises [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Financial Instruments, Owned, Mortgages, Mortgage-backed and Asset-backed Securities, at Fair Value	611,322	421,096
Estimate of Fair Value, Fair Value Disclosure [Member] | Financing Receivable [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securitized mortgage loans, net	61,916	101,116
Estimate of Fair Value, Fair Value Disclosure [Member] | Other Investments [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Investments, Fair Value Disclosure	810	892
Repurchase Agreements [Member] | Carrying (Reported) Amount, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Repurchase Agreements, Fair Value Disclosure	3,564,128	2,093,793
Repurchase Agreements [Member] | Estimate of Fair Value, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Repurchase Agreements, Fair Value Disclosure	3,564,787	2,093,793
Borrowings [Member] | Carrying (Reported) Amount, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt Instrument, Fair Value Disclosure	30,504	70,895
Borrowings [Member] | Estimate of Fair Value, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt Instrument, Fair Value Disclosure	30,756	69,752
Derivative Financial Instruments, Liabilities [Member] | Carrying (Reported) Amount, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative Liabilities	42,537	27,997
Derivative Financial Instruments, Liabilities [Member] | Estimate of Fair Value, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative Liabilities	$ 42,537	$ 27,997