Fair Value of Assets and Liabilities (Narrative) (Details) $ in Thousands	Dec. 31, 2021 USD ($)	Dec. 31, 2021	Dec. 31, 2021 $ / shares	Dec. 31, 2020 USD ($)	Dec. 31, 2019 USD ($)	Dec. 31, 2018 USD ($)
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Available-for-sale securities	$ 2,327,793			$ 3,055,839
Loans held-for-investment	38,598			55,134
MSRs	147,571			92,081	$ 85,638	$ 75,183
Individually assessed loans - foreclosure probable and collateral-dependent	93,270			139,662
Estimate of Fair Value Measurement
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Available-for-sale securities	2,327,793			3,055,839
Remaining contractual principal balance outstanding, mortgage loans held-for-sale	801,600			1,200,000
Non-performing
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage loans held-for-sale, at fair value	125,500			134,100
Appraisal adjustment - cost of sale | Other Real Estate Owned
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Other real estate owned, measurement input		0.10
Measured at fair value on a recurring basis:
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Loans held-for-investment	38,598			55,134
MSRs	147,571			92,081
Mortgage loans held-for-sale, at fair value	817,912			1,272,090
Measured at fair value on a non-recurring basis:
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Individually assessed loans - foreclosure probable and collateral-dependent	63,415
Other real estate owned	4,271
Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Individually assessed loans - foreclosure probable and collateral-dependent	93,300
Level 3 | Portion at Other than Fair Value Measurement
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Individually assessed loans - foreclosure probable and collateral-dependent	29,900
Level 3 | Measured at fair value on a recurring basis:
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Loans held-for-investment	15,891			10,280
MSRs	147,571			92,081
Derivative assets	10,600
Mortgage loans held-for-sale, at fair value	0			0
Level 3 | Measured at fair value on a recurring basis: | Discount rate | Weighted Average | Discounted cash flows
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
MSRs, measurement input		0.0980
Level 3 | Measured at fair value on a recurring basis: | Discount rate | Weighted Average | Loans held-for-investment
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		0.0300
Level 3 | Measured at fair value on a recurring basis: | Discount rate | Weighted Average | MSRs
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		0.0980
Level 3 | Measured at fair value on a recurring basis: | Discount rate | Minimum | Discounted cash flows
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
MSRs, measurement input		0.06
Level 3 | Measured at fair value on a recurring basis: | Discount rate | Minimum | MSRs
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		0.06
Level 3 | Measured at fair value on a recurring basis: | Discount rate | Maximum | Discounted cash flows
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
MSRs, measurement input		0.19
Level 3 | Measured at fair value on a recurring basis: | Discount rate | Maximum | MSRs
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		0.19
Level 3 | Measured at fair value on a recurring basis: | Constant prepayment rate (CPR) | Loans held-for-investment
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		0.1164
Level 3 | Measured at fair value on a recurring basis: | Constant prepayment rate (CPR) | Weighted Average | Discounted cash flows
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
MSRs, measurement input		0.1164
Level 3 | Measured at fair value on a recurring basis: | Constant prepayment rate (CPR) | Weighted Average | MSRs
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		0.1164
Level 3 | Measured at fair value on a recurring basis: | Constant prepayment rate (CPR) | Minimum | Discounted cash flows
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
MSRs, measurement input		0
Level 3 | Measured at fair value on a recurring basis: | Constant prepayment rate (CPR) | Minimum | MSRs
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		0
Level 3 | Measured at fair value on a recurring basis: | Constant prepayment rate (CPR) | Maximum | Discounted cash flows
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
MSRs, measurement input		0.83
Level 3 | Measured at fair value on a recurring basis: | Constant prepayment rate (CPR) | Maximum | MSRs
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		0.83
Level 3 | Measured at fair value on a recurring basis: | Credit discount | Weighted Average | Loans held-for-investment
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		0.0057
Level 3 | Measured at fair value on a recurring basis: | Credit discount | Minimum | Loans held-for-investment
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		0
Level 3 | Measured at fair value on a recurring basis: | Credit discount | Maximum | Loans held-for-investment
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		0.03
Level 3 | Measured at fair value on a recurring basis: | Cost of servicing | Weighted Average | Discounted cash flows
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
MSRs, measurement input		75	75
Level 3 | Measured at fair value on a recurring basis: | Cost of servicing | Minimum | Discounted cash flows
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
MSRs, measurement input | $ / shares			70
Level 3 | Measured at fair value on a recurring basis: | Cost of servicing | Maximum | Discounted cash flows
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
MSRs, measurement input | $ / shares			200
Level 3 | Measured at fair value on a recurring basis: | Cost of servicing - delinquent | Weighted Average | Discounted cash flows
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
MSRs, measurement input		312	312
Level 3 | Measured at fair value on a recurring basis: | Cost of servicing - delinquent | Minimum | Discounted cash flows
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
MSRs, measurement input | $ / shares			200
Level 3 | Measured at fair value on a recurring basis: | Cost of servicing - delinquent | Maximum | Discounted cash flows
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
MSRs, measurement input | $ / shares			1,000
Level 3 | Measured at fair value on a recurring basis: | Pull-through rate | Weighted Average | Discounted cash flows
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivatives, measurement input		0.8578
Level 3 | Measured at fair value on a recurring basis: | Pull-through rate | Minimum | Discounted cash flows
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivatives, measurement input		0.26
Level 3 | Measured at fair value on a recurring basis: | Pull-through rate | Maximum | Discounted cash flows
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivatives, measurement input		1
Level 3 | Measured at fair value on a non-recurring basis:
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Individually assessed loans - foreclosure probable and collateral-dependent	63,415
Other real estate owned	4,271
Municipal
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Available-for-sale securities	165,594			146,910
Municipal | Measured at fair value on a recurring basis:
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Available-for-sale securities	165,594			146,910
Municipal | Level 3 | Measured at fair value on a recurring basis:
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Available-for-sale securities	105,687			109,876
U.S. Government agencies
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Available-for-sale securities	52,507			84,513
U.S. Government agencies | Measured at fair value on a recurring basis:
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Available-for-sale securities	52,507			84,513
U.S. Government agencies | Level 3 | Measured at fair value on a recurring basis:
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Available-for-sale securities	$ 0			$ 1,966