Fair Value of Assets and Liabilities (Narrative) (Details) $ in Thousands	12 Months Ended
	Dec. 31, 2019 USD ($) $ / Loan	Dec. 31, 2018 USD ($)	Dec. 31, 2017 USD ($)	Dec. 31, 2016 USD ($)
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Available-for-sale securities	$ 3,106,214	$ 2,126,081
MSRs	85,638	75,183	$ 33,676	$ 19,103
Mortgage loans held-for-sale	377,313	264,070
Impaired loans	$ 120,045	127,269
Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Pull through rate, derivatives (as a percent)	23.00%
Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Pull through rate, derivatives (as a percent)	100.00%
Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Pull through rate, derivatives (as a percent)	81.86%
Estimate of Fair Value Measurement
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Available-for-sale securities	$ 3,106,214	2,126,081
Principal amount outstanding on loans held-for-sale or securitization or asset-backed financing arrangement	368,000	253,700
Mortgage loans held-for-sale	377,313	264,070
Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	120,000
Level 3 | Valued Using Discounted Cash Flow Model
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	29,700
Municipal
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Available-for-sale securities	145,318	138,490
U.S. Government agencies
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Available-for-sale securities	365,442	140,307
Fair Value, Measurements, Recurring
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Loans held-for-investment	132,718	93,857
MSRs	85,638	75,183
Derivative assets	103,644	73,172
Mortgage loans held-for-sale	377,313	264,070
Fair Value, Measurements, Recurring | Non-performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage loans held-for-sale	1,800	1,900
Fair Value, Measurements, Recurring | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Loans held-for-investment	9,620	11,347
MSRs	85,638	75,183
Derivative assets	$ 2,631	2,457
Fair Value, Measurements, Recurring | Level 3 | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage servicing rights, measurement input	0.07
Fair Value, Measurements, Recurring | Level 3 | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage servicing rights, measurement input	0.17
Fair Value, Measurements, Recurring | Level 3 | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage servicing rights, measurement input	0.0996
Fair Value, Measurements, Recurring | Level 3 | MSRs | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Cost of servicing loan (in dollars per loan) | $ / Loan	77
Cost of servicing delinquent loan (in dollars per loan) | $ / Loan	396
Fair Value, Measurements, Recurring | Municipal
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Available-for-sale securities	$ 145,318	138,490
Fair Value, Measurements, Recurring | Municipal | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Available-for-sale securities	111,950	108,926
Fair Value, Measurements, Recurring | U.S. Government agencies
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Available-for-sale securities	365,442	140,307
Fair Value, Measurements, Recurring | U.S. Government agencies | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Available-for-sale securities	2,646	$ 3,150
Fair Value, Measurements, Nonrecurring
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	90,307
Other real estate owned	15,171
Fair Value, Measurements, Nonrecurring | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Impaired loans	90,307
Other real estate owned	$ 15,171
Discount rate | Fair Value, Measurements, Recurring | Level 3 | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Loans held-for-investment, measurement input	0.03
Discount rate | Fair Value, Measurements, Recurring | Level 3 | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Loans held-for-investment, measurement input	0.04
Discount rate | Fair Value, Measurements, Recurring | Level 3 | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Loans held-for-investment, measurement input	0.0347
Constant prepayment rate (CPR) | Fair Value, Measurements, Recurring | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Loans held-for-investment, measurement input	0.1412
Constant prepayment rate (CPR) | Fair Value, Measurements, Recurring | Level 3 | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage servicing rights, measurement input	0
Constant prepayment rate (CPR) | Fair Value, Measurements, Recurring | Level 3 | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage servicing rights, measurement input	0.94
Constant prepayment rate (CPR) | Fair Value, Measurements, Recurring | Level 3 | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage servicing rights, measurement input	0.1412
Credit discount | Fair Value, Measurements, Recurring | Level 3 | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Loans held-for-investment, measurement input	0
Credit discount | Fair Value, Measurements, Recurring | Level 3 | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Loans held-for-investment, measurement input	0.08
Credit discount | Fair Value, Measurements, Recurring | Level 3 | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Loans held-for-investment, measurement input	0.0137
Appraisal value | Appraisal adjustment - cost of sale | Fair Value, Measurements, Nonrecurring | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Other real estate owned, including covered other real-estate owned, measurement input	0.10
Appraisal value | Appraisal adjustment - cost of sale | Fair Value, Measurements, Nonrecurring | Level 3 | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Other real estate owned, including covered other real-estate owned, measurement input	0.1000