Investment Securities (Details) (USD $)	3 Months Ended		9 Months Ended				0 Months Ended	9 Months Ended	12 Months Ended	3 Months Ended		9 Months Ended		12 Months Ended				3 Months Ended		9 Months Ended		12 Months Ended	3 Months Ended	9 Months Ended		3 Months Ended	9 Months Ended
	Sep. 30, 2013	Sep. 30, 2012	Sep. 30, 2013	Sep. 30, 2012	Dec. 31, 2012	Sep. 30, 2013 CCC-	Sep. 16, 2013 Mortgage-backed securities	Sep. 30, 2013 Mortgage-backed securities	Dec. 31, 2012 Mortgage-backed securities	Sep. 30, 2013 CMBS	Sep. 30, 2012 CMBS	Sep. 30, 2013 CMBS	Sep. 30, 2012 CMBS	Dec. 31, 2012 CMBS	Dec. 31, 2012 CMBS Minimum	Dec. 31, 2012 CMBS Maximum	Dec. 31, 2012 CMBS BB+	Sep. 30, 2013 RMBS	Sep. 30, 2012 RMBS	Sep. 30, 2013 RMBS	Sep. 30, 2012 RMBS	Dec. 31, 2012 RMBS	Sep. 30, 2013 CMBS excluding securities elected as fair value option	Sep. 30, 2013 CMBS excluding securities elected as fair value option	Dec. 31, 2012 CMBS excluding securities elected as fair value option	Sep. 30, 2013 CMBS, fair value option	Sep. 30, 2013 CMBS, fair value option	Sep. 30, 2013 HTM Security	Sep. 30, 2013 Equity Securities	Dec. 31, 2012 Equity Securities
Mortgage-Backed Securities Available-for-Sale
Investment securities ($529,423 and $884,254 held at fair value)	$ 566,793,000		$ 566,793,000		$ 884,254,000					$ 112,436,000		$ 112,436,000		$ 529,434,000				$ 316,261,000		$ 316,261,000		$ 333,153,000				$ 85,710,000	$ 85,710,000	$ 37,370,000	$ 15,016,000	$ 21,700,000
Fair value of investment securities eliminated as a result of consolidation of VIEs																										324,400,000	324,400,000
Purchases	23,871,000	95,820,000	82,754,000	575,690,000						1,889,000		1,889,000	372,252,000						95,820,000	20,090,000	203,438,000					21,982,000	23,601,000	37,174,000
Sales	(206,972,000)	(182,886,000)	(442,877,000)	(199,510,000)						(206,972,000)	(173,461,000)	(413,323,000)	(173,461,000)						(9,425,000)	(12,713,000)	(26,049,000)						(10,072,000)		(6,769,000)
Principal collections	(16,670,000)	(24,416,000)	(56,793,000)	(67,452,000)						(2,546,000)	(5,874,000)	(10,031,000)	(15,142,000)					(14,124,000)	(18,542,000)	(46,762,000)	(52,310,000)
Purchase Amortized Cost								372,917,000	791,385,000	99,081,000		99,081,000		498,064,000				273,836,000		273,836,000		293,321,000
Credit OTTI								(10,573,000)	(10,194,000)
Credit OTTI																		(10,573,000)		(10,573,000)		(10,194,000)
Recorded Amortized Cost								362,344,000	781,191,000	99,081,000		99,081,000		498,064,000				263,263,000		263,263,000		283,127,000
Non-Credit OTTI	(34,000)	(61,000)	(1,007,000)	(2,854,000)				(34,000)
Non-Credit OTTI																		(34,000)		(34,000)
Unrealized Gains								69,745,000	82,087,000	13,355,000		13,355,000		31,370,000				56,390,000		56,390,000		50,717,000
Unrealized Losses								(3,358,000)	(691,000)									(3,358,000)		(3,358,000)		(691,000)
Net Fair Value Adjustment								66,353,000	81,396,000	13,355,000		13,355,000		31,370,000				52,998,000		52,998,000		50,026,000
Fair Value								428,697,000	862,587,000	112,436,000		112,436,000		529,434,000				316,261,000		316,261,000		333,153,000
Fair value of investment securities before consolidation of VIEs																										410,100,000	410,100,000
Proceeds from sales of investment securities			442,877,000	199,510,000			206,900,000
Gain on sale of investments			23,728,000	19,274,000			6,400,000
Weighted Average Coupon (as a percent)										11.50%		11.50%		4.30%				1.00%		1.00%		1.10%				5.60%	5.60%
Weighted Average Life (WAL)												6 years		3 years 3 months 18 days						5 years 10 months 24 days		5 years 4 months 24 days					3 years 4 months 24 days
Description of variable rate basis								one-month LIBOR	one-month LIBOR					LIBOR						LIBOR		LIBOR
Loan-to-value ratio of securities not rated (as a percent)															39.00%	44.00%
Effective variable rate basis (as a percent)								0.17885%	0.2087%
Aggregate participating investment in securities						29,600,000
Amount not rated																										56,100,000	56,100,000
Percentage of CMBS securities																	20.40%
Percentage of securities where obligors are special purpose entities																	79.60%
Principal balance																		452,868,000		452,868,000		489,218,000	99,081,000	99,081,000	519,575,000
Unpiad Principal Balance of investment securities before consolidation of VIEs																										2,900,000,000	2,900,000,000
Purchases in which fair value option was elected																										33,400,000	116,800,000
Purchase amount reflected as repayment of debt of consolidated VIEs																										11,400,000	93,200,000
Accretable yield																		(110,265,000)		(110,265,000)		(108,486,000)			(21,511,000)
Non-accretable difference																		(79,340,000)		(79,340,000)		(97,605,000)
Total discount																		(189,605,000)		(189,605,000)		(206,091,000)			(21,511,000)
Amortized Cost																		263,263,000		263,263,000		283,127,000	99,081,000	99,081,000	498,064,000
Changes to accretable yield
Balance at the beginning of the period																		107,317,000		108,486,000			7,081,000	21,511,000
Accretion of discount																		(5,940,000)		(17,846,000)			(885,000)	(5,442,000)
Purchases																				5,738,000
Sales																				(2,418,000)			(6,196,000)	(16,069,000)
OTTI																		52,000		453,000
Transfer to/from non-accretable difference																		8,836,000		15,852,000
Balance at the end of the period																		110,265,000		110,265,000		108,486,000
Changes to non accretable difference
Balance at the beginning of the period																		89,306,000		97,605,000
Principal write-downs																		(1,130,000)		(2,133,000)
Purchases																				1,758,000
Sales																				(2,038,000)
Transfer to/from non-accretable difference																		(8,836,000)		(15,852,000)
Balance at the end of the period																		79,340,000		79,340,000		97,605,000
Credit deteriorated RMBS																		350,400,000		350,400,000		438,000,000
Accretable yield related to credit deteriorated RMBS																		$ 85,500,000		$ 85,500,000		$ 93,600,000