Commitments and Contingencies - Legal (Details) - USD ($) $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2020	Jun. 30, 2019	Jun. 30, 2020	Jun. 30, 2019	Jun. 30, 2020	Dec. 31, 2019
Representation and Warranty Claims
Roll-forward of claims activity for loans put-back to the mortgage origination segment
Balance, beginning of year	$ 34,779	$ 30,112	$ 32,144	$ 33,784
Claims made	6,141	6,504	12,212	9,686
Claims resolved with no payment	(673)	(1,579)	(1,657)	(7,266)
Repurchases	(5,053)	(1,478)	(7,383)	(2,645)
Indemnification payments		(485)	(122)	(485)
Balance, end of year	35,194	33,074	35,194	33,074
Reserve for Indemnification Liability:
Total	35,194	33,074	35,194	33,074	$ 35,194	$ 32,144
Indemnification Agreement
Commitments and Contingencies
Provision for indemnification losses	3,900	800	4,600	1,300
Roll-forward of claims activity for loans put-back to the mortgage origination segment
Balance, beginning of year	12,148	10,721	11,776	10,701
Additions for new sales	2,897	792	3,622	1,281
Repurchases	(210)	(127)	(480)	(209)
Early payment defaults	(359)	(97)	(402)	(239)
Indemnification payments		(92)	(40)	(95)
Change in reserves for loans sold in prior years	987	(364)	987	(606)
Balance, end of year	15,463	10,833	15,463	10,833
Reserve for Indemnification Liability:
Specific claims					1,423	1,071
Incurred but not reported claims					14,040	10,705
Total	$ 15,463	$ 10,833	$ 11,776	$ 10,833	$ 15,463	$ 11,776