Loans Receivable and Allowance For Loan Losses - Loans by Credit Quality Indicator (Details) $ in Thousands	12 Months Ended
	Dec. 31, 2019 USD ($) loan	Dec. 31, 2018 USD ($)
Financing Receivable, Recorded Investment [Line Items]
Total loans	$ 1,351,893	$ 1,326,238
Number of reported TDR loan modified previous 12 months and subsequently defaulted | loan	0
Number of TDR loans | loan	0
REAL ESTATE LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	$ 824,040	846,323
Commercial
Financing Receivable, Recorded Investment [Line Items]
Total loans	210,749	204,699
Construction and development
Financing Receivable, Recorded Investment [Line Items]
Total loans	179,654	247,306
Home equity.
Financing Receivable, Recorded Investment [Line Items]
Total loans	38,167	40,258
One-to-four-family
Financing Receivable, Recorded Investment [Line Items]
Total loans	261,539	249,397
Multi-family
Financing Receivable, Recorded Investment [Line Items]
Total loans	133,931	104,663
CONSUMER LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	326,210	275,473
Indirect home improvement
Financing Receivable, Recorded Investment [Line Items]
Total loans	210,653	167,793
Solar
Financing Receivable, Recorded Investment [Line Items]
Total loans	44,038	44,433
Marine
Financing Receivable, Recorded Investment [Line Items]
Total loans	67,179	57,822
Other consumer loans
Financing Receivable, Recorded Investment [Line Items]
Total loans	4,340	5,425
COMMERCIAL BUSINESS LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	201,643	204,442
Commercial and industrial
Financing Receivable, Recorded Investment [Line Items]
Total loans	140,531	138,686
Warehouse lending
Financing Receivable, Recorded Investment [Line Items]
Total loans	61,112	65,756
Pass
Financing Receivable, Recorded Investment [Line Items]
Total loans	1,317,980	1,303,884
Pass | REAL ESTATE LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	806,126	839,002
Pass | Commercial
Financing Receivable, Recorded Investment [Line Items]
Total loans	203,703	203,557
Pass | Construction and development
Financing Receivable, Recorded Investment [Line Items]
Total loans	177,109	244,577
Pass | Home equity.
Financing Receivable, Recorded Investment [Line Items]
Total loans	37,942	39,846
Pass | One-to-four-family
Financing Receivable, Recorded Investment [Line Items]
Total loans	259,580	247,575
Pass | Multi-family
Financing Receivable, Recorded Investment [Line Items]
Total loans	127,792	103,447
Pass | CONSUMER LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	325,717	275,037
Pass | Indirect home improvement
Financing Receivable, Recorded Investment [Line Items]
Total loans	210,202	167,426
Pass | Solar
Financing Receivable, Recorded Investment [Line Items]
Total loans	44,021	44,392
Pass | Marine
Financing Receivable, Recorded Investment [Line Items]
Total loans	67,179	57,804
Pass | Other consumer loans
Financing Receivable, Recorded Investment [Line Items]
Total loans	4,315	5,415
Pass | COMMERCIAL BUSINESS LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	186,137	189,845
Pass | Commercial and industrial
Financing Receivable, Recorded Investment [Line Items]
Total loans	125,025	124,089
Pass | Warehouse lending
Financing Receivable, Recorded Investment [Line Items]
Total loans	61,112	65,756
Watch
Financing Receivable, Recorded Investment [Line Items]
Total loans	22,028	14,107
Watch | REAL ESTATE LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	11,593	5,294
Watch | Commercial
Financing Receivable, Recorded Investment [Line Items]
Total loans	2,274	1,142
Watch | Construction and development
Financing Receivable, Recorded Investment [Line Items]
Total loans	2,545	2,729
Watch | Home equity.
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Watch | One-to-four-family
Financing Receivable, Recorded Investment [Line Items]
Total loans	635	207
Watch | Multi-family
Financing Receivable, Recorded Investment [Line Items]
Total loans	6,139	1,216
Watch | CONSUMER LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Watch | Indirect home improvement
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Watch | Solar
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Watch | Marine
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Watch | Other consumer loans
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Watch | COMMERCIAL BUSINESS LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	10,435	8,813
Watch | Commercial and industrial
Financing Receivable, Recorded Investment [Line Items]
Total loans	10,435	8,813
Watch | Warehouse lending
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total loans	5,223	254
Special Mention | REAL ESTATE LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	3,781	246
Special Mention | Commercial
Financing Receivable, Recorded Investment [Line Items]
Total loans	3,686	0
Special Mention | Construction and development
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Special Mention | Home equity.
Financing Receivable, Recorded Investment [Line Items]
Total loans	35	183
Special Mention | One-to-four-family
Financing Receivable, Recorded Investment [Line Items]
Total loans	60	63
Special Mention | Multi-family
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Special Mention | CONSUMER LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	8
Special Mention | Indirect home improvement
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Special Mention | Solar
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Special Mention | Marine
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Special Mention | Other consumer loans
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	8
Special Mention | COMMERCIAL BUSINESS LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	1,442	0
Special Mention | Commercial and industrial
Financing Receivable, Recorded Investment [Line Items]
Total loans	1,442	0
Special Mention | Warehouse lending
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Substandard
Financing Receivable, Recorded Investment [Line Items]
Total loans	6,662	7,993
Substandard | REAL ESTATE LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	2,540	1,781
Substandard | Commercial
Financing Receivable, Recorded Investment [Line Items]
Total loans	1,086	0
Substandard | Construction and development
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Substandard | Home equity.
Financing Receivable, Recorded Investment [Line Items]
Total loans	190	229
Substandard | One-to-four-family
Financing Receivable, Recorded Investment [Line Items]
Total loans	1,264	1,552
Substandard | Multi-family
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Substandard | CONSUMER LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	493	428
Substandard | Indirect home improvement
Financing Receivable, Recorded Investment [Line Items]
Total loans	451	367
Substandard | Solar
Financing Receivable, Recorded Investment [Line Items]
Total loans	17	41
Substandard | Marine
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	18
Substandard | Other consumer loans
Financing Receivable, Recorded Investment [Line Items]
Total loans	25	2
Substandard | COMMERCIAL BUSINESS LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	3,629	5,784
Substandard | Commercial and industrial
Financing Receivable, Recorded Investment [Line Items]
Total loans	3,629	5,784
Substandard | Warehouse lending
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Doubtful
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Doubtful | REAL ESTATE LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Doubtful | Commercial
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Doubtful | Construction and development
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Doubtful | Home equity.
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Doubtful | One-to-four-family
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Doubtful | Multi-family
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Doubtful | CONSUMER LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Doubtful | Indirect home improvement
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Doubtful | Solar
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Doubtful | Marine
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Doubtful | Other consumer loans
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Doubtful | COMMERCIAL BUSINESS LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Doubtful | Commercial and industrial
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Doubtful | Warehouse lending
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Loss
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Loss | REAL ESTATE LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Loss | Commercial
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Loss | Construction and development
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Loss | Home equity.
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Loss | One-to-four-family
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Loss | Multi-family
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Loss | CONSUMER LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Loss | Indirect home improvement
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Loss | Solar
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Loss | Marine
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Loss | Other consumer loans
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Loss | COMMERCIAL BUSINESS LOANS.
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Loss | Commercial and industrial
Financing Receivable, Recorded Investment [Line Items]
Total loans	0	0
Loss | Warehouse lending
Financing Receivable, Recorded Investment [Line Items]
Total loans	$ 0	$ 0