Loans Receivable - Past Due Loans (Details) - USD ($)	Dec. 31, 2019	Dec. 31, 2018
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	$ 25,393,000	$ 14,113,000
Current	3,721,932,000	3,611,631,000
Gross Loan and Lease Receivable excluding PCI Loans	3,747,325,000	3,625,744,000
Certain Loans and Debt Securities Acquired in Transfer, with Related Allowance for Credit Losses Due to Subsequent Impairment	18,113,000	24,907,000
Loans and Leases Receivable, Gross	3,765,438,000	3,650,651,000
90 Days or More and Still Accruing	0	0
30-89 Days
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	15,970,000	10,087,000
90 Days or Greater
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	9,423,000	4,026,000
Commercial business
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	21,060,000	10,428,000
Current	2,916,203,000	2,909,748,000
Gross Loan and Lease Receivable excluding PCI Loans	2,937,263,000	2,920,176,000
Certain Loans and Debt Securities Acquired in Transfer, with Related Allowance for Credit Losses Due to Subsequent Impairment	12,772,000	17,707,000
Loans and Leases Receivable, Gross	2,950,035,000	2,937,883,000
Commercial business | 30-89 Days
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	11,637,000	6,636,000
Commercial business | 90 Days or Greater
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	9,423,000	3,792,000
Commercial business | Commercial and industrial
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	17,250,000	4,992,000
Current	832,217,000	845,181,000
Gross Loan and Lease Receivable excluding PCI Loans	849,467,000	850,173,000
Certain Loans and Debt Securities Acquired in Transfer, with Related Allowance for Credit Losses Due to Subsequent Impairment	2,367,000	3,433,000
Loans and Leases Receivable, Gross	851,834,000	853,606,000
Commercial business | Commercial and industrial | 30-89 Days
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	10,476,000	2,711,000
Commercial business | Commercial and industrial | 90 Days or Greater
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	6,774,000	2,281,000
Commercial business | Owner-occupied commercial real estate
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	1,413,000	921,000
Current	800,282,000	771,678,000
Gross Loan and Lease Receivable excluding PCI Loans	801,695,000	772,599,000
Certain Loans and Debt Securities Acquired in Transfer, with Related Allowance for Credit Losses Due to Subsequent Impairment	4,914,000	7,215,000
Loans and Leases Receivable, Gross	806,609,000	779,814,000
Commercial business | Owner-occupied commercial real estate | 30-89 Days
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	607,000	513,000
Commercial business | Owner-occupied commercial real estate | 90 Days or Greater
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	806,000	408,000
Commercial business | Non-owner occupied commercial real estate
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	2,397,000	4,515,000
Current	1,283,704,000	1,292,889,000
Gross Loan and Lease Receivable excluding PCI Loans	1,286,101,000	1,297,404,000
Certain Loans and Debt Securities Acquired in Transfer, with Related Allowance for Credit Losses Due to Subsequent Impairment	5,491,000	7,059,000
Loans and Leases Receivable, Gross	1,291,592,000	1,304,463,000
Commercial business | Non-owner occupied commercial real estate | 30-89 Days
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	554,000	3,412,000
Commercial business | Non-owner occupied commercial real estate | 90 Days or Greater
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	1,843,000	1,103,000
One-to-four family residential
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	797,000	227,000
Current	127,712,000	98,221,000
Gross Loan and Lease Receivable excluding PCI Loans	128,509,000	98,448,000
Certain Loans and Debt Securities Acquired in Transfer, with Related Allowance for Credit Losses Due to Subsequent Impairment	3,579,000	3,315,000
Loans and Leases Receivable, Gross	132,088,000	101,763,000
One-to-four family residential | 30-89 Days
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	797,000	227,000
One-to-four family residential | 90 Days or Greater
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	0	0
Real estate construction and land development
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	1,516,000	899,000
Current	275,171,000	214,138,000
Gross Loan and Lease Receivable excluding PCI Loans	276,687,000	215,037,000
Certain Loans and Debt Securities Acquired in Transfer, with Related Allowance for Credit Losses Due to Subsequent Impairment	0	423,000
Loans and Leases Receivable, Gross	276,687,000	215,460,000
Real estate construction and land development | 30-89 Days
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	1,516,000	665,000
Real estate construction and land development | 90 Days or Greater
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	0	234,000
Real estate construction and land development | One-to-four family residential
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	1,516,000	899,000
Current	103,394,000	101,451,000
Gross Loan and Lease Receivable excluding PCI Loans	104,910,000	102,350,000
Certain Loans and Debt Securities Acquired in Transfer, with Related Allowance for Credit Losses Due to Subsequent Impairment	0	380,000
Loans and Leases Receivable, Gross	104,910,000	102,730,000
Real estate construction and land development | One-to-four family residential | 30-89 Days
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	1,516,000	665,000
Real estate construction and land development | One-to-four family residential | 90 Days or Greater
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	0	234,000
Real estate construction and land development | Five or more family residential and commercial properties
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	0	0
Current	171,777,000	112,687,000
Gross Loan and Lease Receivable excluding PCI Loans	171,777,000	112,687,000
Certain Loans and Debt Securities Acquired in Transfer, with Related Allowance for Credit Losses Due to Subsequent Impairment	0	43,000
Loans and Leases Receivable, Gross	171,777,000	112,730,000
Real estate construction and land development | Five or more family residential and commercial properties | 30-89 Days
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	0	0
Real estate construction and land development | Five or more family residential and commercial properties | 90 Days or Greater
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	0	0
Consumer
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	2,020,000	2,559,000
Current	402,846,000	389,524,000
Gross Loan and Lease Receivable excluding PCI Loans	404,866,000	392,083,000
Certain Loans and Debt Securities Acquired in Transfer, with Related Allowance for Credit Losses Due to Subsequent Impairment	1,762,000	3,462,000
Loans and Leases Receivable, Gross	406,628,000	395,545,000
Consumer | 30-89 Days
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	2,020,000	2,559,000
Consumer | 90 Days or Greater
Financing Receivable, Recorded Investment, Past Due [Line Items]
Total Past Due	$ 0	$ 0