Loans Held for Investment - Schedule of Past Due Status of Loans Held for Investment, Gross (Details) - USD ($) $ in Thousands	6 Months Ended	12 Months Ended
	Dec. 31, 2020	Jun. 30, 2020	Dec. 31, 2019
Financing Receivable, Recorded Investment, Past Due [Line Items]
Loans deemed uncollectible, period of delinquency	90 days	90 days
Current	$ 845,548	$ 898,947
30-89 Days Past Due	350	219
Non-Accrual	11,182	5,300
Total Loans Held for Investment, Gross	857,080	904,466	$ 941,740
Mortgage loans, Single-family
Financing Receivable, Recorded Investment, Past Due [Line Items]
Current	246,334	293,326
30-89 Days Past Due	348	219
Non-Accrual	11,182	5,265
Total Loans Held for Investment, Gross	257,864	298,810	347,344
Mortgage Loans, Multi Family
Financing Receivable, Recorded Investment, Past Due [Line Items]
Current	488,412	491,903
30-89 Days Past Due	0	0
Non-Accrual	0	0
Total Loans Held for Investment, Gross	488,412	491,903	479,151
Mortgage loans, Commercial real estate
Financing Receivable, Recorded Investment, Past Due [Line Items]
Current	102,551	105,235
30-89 Days Past Due	0	0
Non-Accrual	0	0
Total Loans Held for Investment, Gross	102,551	105,235	107,613
Construction
Financing Receivable, Recorded Investment, Past Due [Line Items]
Current	7,135	7,801
30-89 Days Past Due	0	0
Non-Accrual	0	0
Total Loans Held for Investment, Gross	7,135	7,801	6,914
Mortgage Loans Other
Financing Receivable, Recorded Investment, Past Due [Line Items]
Current	141	143
30-89 Days Past Due	0	0
Non-Accrual	0	0
Total Loans Held for Investment, Gross	141	143	578
Commercial business loans
Financing Receivable, Recorded Investment, Past Due [Line Items]
Current	882	445
30-89 Days Past Due	0	0
Non-Accrual	0	35
Total Loans Held for Investment, Gross	882	480	578
Consumer loans
Financing Receivable, Recorded Investment, Past Due [Line Items]
Current	93	94
30-89 Days Past Due	2	0
Non-Accrual	0	0
Total Loans Held for Investment, Gross	$ 95	$ 94	$ 140
Bankruptcy [Member]
Financing Receivable, Recorded Investment, Past Due [Line Items]
Loans deemed uncollectible, period of delinquency	60 days
Troubled Debt Restructurings [Member]
Financing Receivable, Recorded Investment, Past Due [Line Items]
Loans deemed uncollectible, period of delinquency	90 days
Commercial Real Estate Or Second Mortgage [Member]
Financing Receivable, Recorded Investment, Past Due [Line Items]
Loans deemed uncollectible, period of delinquency	120 days