Netting of Financial Instruments (Details 2) (USD $) In Thousands, unless otherwise specified	Sep. 30, 2013	Dec. 31, 2012
Derivatives: Subject to a master netting arrangement:
Gross amounts of recognized liabilities	$ 48,284	$ 2,283
Netting	(42,667)	(1,785)
Net amounts of assets presented in the balance sheet	5,617	498
Total derivatives
Gross amounts of recognized liabilities	48,443	2,294
Net amounts of liabilities presented in the consolidated balance sheet	5,776	509
Total
Gross amounts of recognized liabilities	436,326	395,828
Gross amounts offset in the consolidated balance sheet	(42,667)	(1,785)
Net amounts of liabilities presented in the consolidated balance sheet	393,659	394,043
Gross amounts not offset in the balance sheet
Financial instruments	(387,883)	(393,534)
Net amount	5,776	509
Bank of America, N.A.
Total
Net amounts of liabilities presented in the consolidated balance sheet	200,074	150,082
Gross amounts not offset in the balance sheet
Financial instruments	(199,423)	(150,082)
Net amount	651
Credit Suisse First Boston Mortgage Capital LLC
Total
Net amounts of liabilities presented in the consolidated balance sheet	188,645	122,443
Gross amounts not offset in the balance sheet
Financial instruments	(188,460)	(122,252)
Net amount	185	191
Daiwa Capital Markets
Total
Net amounts of liabilities presented in the consolidated balance sheet	1,589	20
Gross amounts not offset in the balance sheet
Net amount	1,589	20
Morgan Stanley Bank, N.A.
Total
Net amounts of liabilities presented in the consolidated balance sheet	543	53
Gross amounts not offset in the balance sheet
Net amount	543	53
Bank of NY Mellon
Total
Net amounts of liabilities presented in the consolidated balance sheet	524
Gross amounts not offset in the balance sheet
Net amount	524
Citibank, N.A.
Total
Net amounts of liabilities presented in the consolidated balance sheet	34	121,200
Gross amounts not offset in the balance sheet
Financial instruments		(121,200)
Net amount	34
Other
Total
Net amounts of liabilities presented in the consolidated balance sheet	2,091	234
Gross amounts not offset in the balance sheet
Net amount	2,091	234
Mortgage loans
Mortgage loans sold under agreements to repurchase
Gross amounts of recognized liabilities	387,883	393,534
Net amounts of liabilities presented in the consolidated balance sheet	387,883	393,534
Forward contracts | Purchases
Derivatives: Subject to a master netting arrangement:
Gross amounts of recognized liabilities	215	389
Forward contracts | Sales
Derivatives: Subject to a master netting arrangement:
Gross amounts of recognized liabilities	48,069	1,894
IRLCs
Derivatives not subject to a master netting arrangement
Gross amounts of recognized liabilities	159	11
Total
Net amounts of liabilities presented in the consolidated balance sheet	159	11
Gross amounts not offset in the balance sheet
Net amount	$ 159	$ 11