Fair Value of Financial Instruments Level 3 (Details) $ in Thousands	3 Months Ended
	Mar. 31, 2021 USD ($) percent
Measurement Input, Prepayment Rate
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Debt Securities, Available-for-sale, Measurement Input | percent	0.10
Loans Held-for-sale, Measurement Input | percent	0.10
Measurement Input, Default Rate
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Debt Securities, Available-for-sale, Measurement Input | percent	0.02
Loans Held-for-sale, Measurement Input | percent	0.02
Measurement Input, Loss Severity
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Debt Securities, Available-for-sale, Measurement Input | percent	0.20
Loans Held-for-sale, Measurement Input | percent	0.20
Measurement Input, Discount Rate
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Debt Securities, Available-for-sale, Measurement Input | percent	0.070
Loans Held-for-sale, Measurement Input | percent	0.070
Non-Agency MBS
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at the beginning of the period	$ 1,275
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Gain (Loss) Included in Other Comprehensive Income (Loss)	(57)
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Settlements	(145)
Balance at the end of the period	1,143
Loans Receivable
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at the beginning of the period	6,264
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Gain (Loss) Included in Earnings	22
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Settlements	(488)
Balance at the end of the period	5,749
Interest Income | Non-Agency MBS
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Accretion (amortization)	70
Interest Income | Loans Receivable
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Accretion (amortization)	$ (5)