Segments - Product Concentration (Details) - USD ($) $ in Thousands	Dec. 31, 2015	Sep. 30, 2015	Jun. 30, 2015	Mar. 31, 2015	Dec. 31, 2014	Sep. 30, 2014	Jun. 30, 2014	Mar. 31, 2014	Dec. 31, 2013
Product concentration
Unpaid principal balance of loans	$ 50,212,264	$ 47,794,561	$ 47,713,739	$ 46,066,660	$ 44,031,890	$ 41,219,196	$ 39,793,733	$ 38,908,865	$ 38,937,027
Fannie Mae DUS program
Product concentration
Unpaid principal balance of loans	22,915,088				20,521,425				19,352,880
Freddie Mac
Product concentration
Unpaid principal balance of loans	17,810,007				12,916,705				10,271,732
Ginnie Mae-HUD
Product concentration
Unpaid principal balance of loans	5,657,809				5,828,981				5,044,193
Life insurance companies and other
Product concentration
Unpaid principal balance of loans	$ 3,829,360				$ 4,764,779				$ 4,268,222