Fair Value of Financial Instruments Recorded basis and Fair Value (Details) (USD $) In Thousands, unless otherwise specified	Mar. 31, 2013	Dec. 31, 2012
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities, Fair Value Disclosure	$ 4,267,917	$ 4,103,981
Derivative Liabilities, Fair Value Disclosure	37,687	42,537
Repurchase Agreements [Member] | Carrying (Reported) Amount, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Repurchase Agreements, Fair Value Disclosure	3,708,255	3,564,128
Repurchase Agreements [Member] | Estimate of Fair Value, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Repurchase Agreements, Fair Value Disclosure	3,708,811	3,564,787
Non-recourse collateralized financing [Member] | Carrying (Reported) Amount, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt Instrument, Fair Value Disclosure	28,473	30,504
Non-recourse collateralized financing [Member] | Estimate of Fair Value, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Debt Instrument, Fair Value Disclosure	28,810	30,756
Derivative Financial Instruments, Liabilities [Member] | Carrying (Reported) Amount, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative Liabilities, Fair Value Disclosure	37,687	42,537
Derivative Financial Instruments, Liabilities [Member] | Estimate of Fair Value, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative Liabilities, Fair Value Disclosure	37,687	42,537
Agency MBS [Member] | Carrying (Reported) Amount, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities, Fair Value Disclosure	3,848,434	3,492,659
Agency MBS [Member] | Estimate of Fair Value, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities, Fair Value Disclosure	3,848,434	3,492,659
Non-Agency MBS [Member] | Carrying (Reported) Amount, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities, Fair Value Disclosure	617,261	611,322
Non-Agency MBS [Member] | Estimate of Fair Value, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities, Fair Value Disclosure	617,261	611,322
Securitized mortgage loans, net [Member] | Carrying (Reported) Amount, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securitized mortgage loans, Fair Value Disclosure	64,825	70,823
Securitized mortgage loans, net [Member] | Estimate of Fair Value, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Securitized mortgage loans, Fair Value Disclosure	54,757	61,916
Other Investments [Member] | Carrying (Reported) Amount, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Other Investments, Fair Value Disclosure	822	858
Other Investments [Member] | Estimate of Fair Value, Fair Value Disclosure [Member] | Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Other Investments, Fair Value Disclosure	$ 775	$ 810