Commitments and Contingencies - Legal (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019	Mar. 31, 2020	Dec. 31, 2019
Representation and Warranty Claims
Roll-forward of claims activity for loans put-back to the mortgage origination segment
Balance, beginning of year	$ 32,144	$ 33,784
Claims made	6,071	3,182
Claims resolved with no payment	(984)	(5,687)
Repurchases	(2,330)	(1,167)
Indemnification payments	(122)
Balance, end of year	34,779	30,112
Reserve for Indemnification Liability:
Total	34,779	30,112	$ 34,779	$ 32,144
Indemnification Agreement
Commitments and Contingencies
Provision for indemnification losses	700	500
Roll-forward of claims activity for loans put-back to the mortgage origination segment
Balance, beginning of year	11,776	10,701
Additions for new sales	725	489
Repurchases	(271)	(82)
Early payment defaults	(39)	(142)
Indemnification payments	(43)	(3)
Change in reserves for loans sold in prior years		(242)
Balance, end of year	12,148	10,721
Reserve for Indemnification Liability:
Specific claims			1,186	1,071
Incurred but not reported claims			10,962	10,705
Total	$ 11,776	$ 10,721	$ 12,148	$ 11,776