Receivables and Other Assets (Details) $ in Thousands	3 Months Ended	12 Months Ended
	Mar. 31, 2017 USD ($) Loan	Dec. 31, 2016 USD ($) Loan
Receivables and Other Assets [Abstract]
Excess servicing income receivable	$ 2,000	$ 5,598
Servicing advances	3,063	1,432
Interest receivable	2,741	2,069
Repurchased loans held for sale	474	1,570
Other receivables	2,007	1,628
Total other assets	$ 10,285	$ 12,297
Number of purchased loans | Loan	0	5
Mortgage loans with an outstanding principal balance		$ 1,640
Number of loans sold | Loan	4