Fair Value of Assets and Liabilities (Summary of Changes in Level Three Assets and Liabilities Measured at Fair Value on a Recurring Basis) (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2019	Dec. 31, 2018
Loans held-for-investment
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Beginning Balance	$ 11,347	$ 33,717
Total net gains (losses) included in net income	827	(1,077)
Total net gains (losses) included in other comprehensive income (loss)	0	0
Issuances	0
Sales	0
Settlements	(5,447)	(28,367)
Net transfers into/(out of) Level 3	2,893	7,074
Ending Balance	9,620	11,347
MSRs
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Beginning Balance	75,183	33,676
Total net gains (losses) included in net income	10,047	27,701
Purchases	408	13,806
Issuances	0
Sales	0
Ending Balance	85,638	75,183
Derivative assets
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Beginning Balance	2,457	2,157
Total net gains (losses) included in net income	174	300
Total net gains (losses) included in other comprehensive income (loss)	0
Issuances	0
Sales	0
Ending Balance	2,631	2,457
Municipal
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Beginning Balance	108,926	77,181
Total net gains (losses) included in other comprehensive income (loss)	3,147	(8,541)
Purchases	38,686	63,644
Issuances	0
Sales	0
Settlements	(38,809)	(23,358)
Ending Balance	111,950	108,926
U.S. Government Agencies
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Beginning Balance	3,150	3,779
Total net gains (losses) included in other comprehensive income (loss)	126	(314)
Issuances	0
Sales	0
Settlements	(630)	(315)
Ending Balance	$ 2,646	$ 3,150