Loans Receivable and the Allowance for Credit Losses - Loans by Credit Quality Indicator (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Financing Receivable, Recorded Investment
Total loans held for investment, net	$ 3,374,575	$ 3,422,187
Loans, gross	3,425,762	3,451,266
Pass
Financing Receivable, Recorded Investment
Loans, gross		3,246,524
Special mention / watch
Financing Receivable, Recorded Investment
Loans, gross		121,709
Substandard
Financing Receivable, Recorded Investment
Loans, gross		82,813
Doubtful
Financing Receivable, Recorded Investment
Loans, gross		184
Loss
Financing Receivable, Recorded Investment
Loans, gross		36
Agricultural
Financing Receivable, Recorded Investment
2020	18,482
2019	18,497
2018	9,635
2017	3,852
2016	4,288
Prior	6,350
Revolving Loans	84,331
Total loans held for investment, net	145,435
Loans, gross	145,435	140,446
Agricultural | Pass
Financing Receivable, Recorded Investment
2020	14,609
2019	15,786
2018	8,035
2017	3,272
2016	3,135
Prior	3,220
Revolving Loans	74,018
Total loans held for investment, net	122,075
Loans, gross		117,374
Agricultural | Special mention / watch
Financing Receivable, Recorded Investment
2020	3,749
2019	1,268
2018	591
2017	404
2016	897
Prior	344
Revolving Loans	5,543
Total loans held for investment, net	12,796
Loans, gross		13,292
Agricultural | Substandard
Financing Receivable, Recorded Investment
2020	124
2019	1,443
2018	1,009
2017	176
2016	256
Prior	2,786
Revolving Loans	4,770
Total loans held for investment, net	10,564
Loans, gross		9,780
Agricultural | Doubtful
Financing Receivable, Recorded Investment
2020	0
Loans, gross		0
Agricultural | Loss
Financing Receivable, Recorded Investment
Loans, gross		0
Commercial and industrial
Financing Receivable, Recorded Investment
2020	64,852
2019	141,241
2018	102,903
2017	119,327
2016	57,268
Prior	135,236
Revolving Loans	243,875
Total loans held for investment, net	864,702
Loans, gross	864,702	835,236
Commercial and industrial | Pass
Financing Receivable, Recorded Investment
2020	62,134
2019	135,839
2018	99,140
2017	113,934
2016	54,861
Prior	127,025
Revolving Loans	227,607
Total loans held for investment, net	820,540
Loans, gross		794,526
Commercial and industrial | Special mention / watch
Financing Receivable, Recorded Investment
2020	334
2019	2,143
2018	2,717
2017	4,012
2016	990
Prior	5,545
Revolving Loans	9,814
Total loans held for investment, net	25,555
Loans, gross		19,038
Commercial and industrial | Substandard
Financing Receivable, Recorded Investment
2020	2,384
2019	3,259
2018	1,046
2017	1,381
2016	1,417
Prior	2,666
Revolving Loans	6,454
Total loans held for investment, net	18,607
Loans, gross		21,635
Commercial and industrial | Doubtful
Financing Receivable, Recorded Investment
2020	0
Loans, gross		1
Commercial and industrial | Loss
Financing Receivable, Recorded Investment
Loans, gross		36
Commercial real estate
Financing Receivable, Recorded Investment
Loans, gross	1,780,446	1,814,859
Commercial real estate | Pass
Financing Receivable, Recorded Investment
Loans, gross		1,687,570
Commercial real estate | Special mention / watch
Financing Receivable, Recorded Investment
Loans, gross		83,511
Commercial real estate | Substandard
Financing Receivable, Recorded Investment
Loans, gross		43,778
Commercial real estate | Doubtful
Financing Receivable, Recorded Investment
Loans, gross		0
Commercial real estate | Loss
Financing Receivable, Recorded Investment
Loans, gross		0
Commercial real estate | Construction & development
Financing Receivable, Recorded Investment
2020	20,568
2019	139,612
2018	44,614
2017	34,407
2016	2,401
Prior	2,156
Revolving Loans	39,163
Total loans held for investment, net	282,921
Loans, gross	282,921	298,077
Commercial real estate | Construction & development | Pass
Financing Receivable, Recorded Investment
2020	18,964
2019	136,833
2018	44,024
2017	34,407
2016	2,285
Prior	2,076
Revolving Loans	29,539
Total loans held for investment, net	268,128
Loans, gross		283,921
Commercial real estate | Construction & development | Special mention / watch
Financing Receivable, Recorded Investment
2020	989
2019	800
2018	590
Prior	35
Revolving Loans	9,059
Total loans held for investment, net	11,473
Loans, gross		11,423
Commercial real estate | Construction & development | Substandard
Financing Receivable, Recorded Investment
2020	615
2019	1,979
2016	116
Prior	45
Revolving Loans	565
Total loans held for investment, net	3,320
Loans, gross		2,733
Commercial real estate | Construction & development | Doubtful
Financing Receivable, Recorded Investment
2020	0
Loans, gross		0
Commercial real estate | Construction & development | Loss
Financing Receivable, Recorded Investment
Loans, gross		0
Commercial real estate | Farmland
Financing Receivable, Recorded Investment
2020	13,639
2019	50,298
2018	25,258
2017	24,251
2016	13,795
Prior	36,741
Revolving Loans	4,795
Total loans held for investment, net	168,777
Loans, gross	168,777	181,885
Commercial real estate | Farmland | Pass
Financing Receivable, Recorded Investment
2020	10,749
2019	38,099
2018	19,395
2017	20,402
2016	12,113
Prior	28,798
Revolving Loans	3,521
Total loans held for investment, net	133,077
Loans, gross		141,107
Commercial real estate | Farmland | Special mention / watch
Financing Receivable, Recorded Investment
2020	1,672
2019	7,034
2018	1,973
2017	1,998
2016	1,255
Prior	1,296
Revolving Loans	703
Total loans held for investment, net	15,931
Loans, gross		21,307
Commercial real estate | Farmland | Substandard
Financing Receivable, Recorded Investment
2020	1,218
2019	5,165
2018	3,890
2017	1,851
2016	427
Prior	6,647
Revolving Loans	571
Total loans held for investment, net	19,769
Loans, gross		19,471
Commercial real estate | Farmland | Doubtful
Financing Receivable, Recorded Investment
2020	0
Loans, gross		0
Commercial real estate | Farmland | Loss
Financing Receivable, Recorded Investment
Loans, gross		0
Commercial real estate | Multifamily
Financing Receivable, Recorded Investment
2020	50,291
2019	22,638
2018	23,646
2017	25,045
2016	21,972
Prior	59,026
Revolving Loans	14,490
Total loans held for investment, net	217,108
Loans, gross	217,108	227,407
Commercial real estate | Multifamily | Pass
Financing Receivable, Recorded Investment
2020	50,291
2019	21,562
2018	23,646
2017	25,045
2016	21,887
Prior	59,026
Revolving Loans	14,490
Total loans held for investment, net	215,947
Loans, gross		226,124
Commercial real estate | Multifamily | Special mention / watch
Financing Receivable, Recorded Investment
2020	0
2016	85
Total loans held for investment, net	85
Loans, gross		90
Commercial real estate | Multifamily | Substandard
Financing Receivable, Recorded Investment
2020	0
2019	1,076
Total loans held for investment, net	1,076
Loans, gross		1,193
Commercial real estate | Multifamily | Doubtful
Financing Receivable, Recorded Investment
2020	0
Loans, gross		0
Commercial real estate | Multifamily | Loss
Financing Receivable, Recorded Investment
Loans, gross		0
Commercial real estate | Commercial real estate-other
Financing Receivable, Recorded Investment
2020	116,727
2019	240,050
2018	127,011
2017	198,062
2016	153,871
Prior	215,265
Revolving Loans	60,654
Total loans held for investment, net	1,111,640
Loans, gross	1,111,640	1,107,490
Commercial real estate | Commercial real estate-other | Pass
Financing Receivable, Recorded Investment
2020	115,059
2019	216,822
2018	113,410
2017	186,570
2016	148,073
Prior	201,195
Revolving Loans	60,246
Total loans held for investment, net	1,041,375
Loans, gross		1,036,418
Commercial real estate | Commercial real estate-other | Special mention / watch
Financing Receivable, Recorded Investment
2020	1,668
2019	21,535
2018	6,378
2017	7,060
2016	4,520
Prior	10,027
Revolving Loans	273
Total loans held for investment, net	51,461
Loans, gross		50,691
Commercial real estate | Commercial real estate-other | Substandard
Financing Receivable, Recorded Investment
2020	0
2019	1,693
2018	7,223
2017	4,432
2016	1,278
Prior	4,043
Revolving Loans	135
Total loans held for investment, net	18,804
Loans, gross		20,381
Commercial real estate | Commercial real estate-other | Doubtful
Financing Receivable, Recorded Investment
2020	0
Loans, gross		0
Commercial real estate | Commercial real estate-other | Loss
Financing Receivable, Recorded Investment
Loans, gross		0
Residential real estate
Financing Receivable, Recorded Investment
Loans, gross	554,290	577,799
Residential real estate | Pass
Financing Receivable, Recorded Investment
Loans, gross		564,404
Residential real estate | Special mention / watch
Financing Receivable, Recorded Investment
Loans, gross		5,829
Residential real estate | Substandard
Financing Receivable, Recorded Investment
Loans, gross		7,402
Residential real estate | Doubtful
Financing Receivable, Recorded Investment
Loans, gross		164
Residential real estate | Loss
Financing Receivable, Recorded Investment
Loans, gross		0
Residential real estate | One- to four- family first liens
Financing Receivable, Recorded Investment
2020	32,556
2019	76,812
2018	60,195
2017	56,952
2016	49,928
Prior	103,791
Revolving Loans	8,821
Total loans held for investment, net	389,055
Loans, gross	389,055	407,418
Residential real estate | One- to four- family first liens | Performing
Financing Receivable, Recorded Investment
2020	32,556
2019	76,673
2018	59,926
2017	56,374
2016	49,581
Prior	101,470
Revolving Loans	8,805
Total loans held for investment, net	385,385
Residential real estate | One- to four- family first liens | Nonperforming
Financing Receivable, Recorded Investment
2020	0
2019	139
2018	269
2017	578
2016	347
Prior	2,321
Revolving Loans	16
Total loans held for investment, net	3,670
Residential real estate | One- to four- family first liens | Pass
Financing Receivable, Recorded Investment
Loans, gross		396,175
Residential real estate | One- to four- family first liens | Special mention / watch
Financing Receivable, Recorded Investment
Loans, gross		4,547
Residential real estate | One- to four- family first liens | Substandard
Financing Receivable, Recorded Investment
Loans, gross		6,532
Residential real estate | One- to four- family first liens | Doubtful
Financing Receivable, Recorded Investment
Loans, gross		164
Residential real estate | One- to four- family first liens | Loss
Financing Receivable, Recorded Investment
Loans, gross		0
Residential real estate | One- to four- family junior liens
Financing Receivable, Recorded Investment
2020	2,215
2019	16,413
2018	19,152
2017	9,474
2016	5,101
Prior	9,334
Revolving Loans	103,546
Total loans held for investment, net	165,235
Loans, gross	165,235	170,381
Residential real estate | One- to four- family junior liens | Performing
Financing Receivable, Recorded Investment
2020	2,215
2019	16,329
2018	19,152
2017	9,441
2016	4,848
Prior	9,022
Revolving Loans	103,524
Total loans held for investment, net	164,531
Residential real estate | One- to four- family junior liens | Nonperforming
Financing Receivable, Recorded Investment
2020	0
2019	84
2018	0
2017	33
2016	253
Prior	312
Revolving Loans	22
Total loans held for investment, net	704
Residential real estate | One- to four- family junior liens | Pass
Financing Receivable, Recorded Investment
Loans, gross		168,229
Residential real estate | One- to four- family junior liens | Special mention / watch
Financing Receivable, Recorded Investment
Loans, gross		1,282
Residential real estate | One- to four- family junior liens | Substandard
Financing Receivable, Recorded Investment
Loans, gross		870
Residential real estate | One- to four- family junior liens | Doubtful
Financing Receivable, Recorded Investment
Loans, gross		0
Residential real estate | One- to four- family junior liens | Loss
Financing Receivable, Recorded Investment
Loans, gross		0
Consumer
Financing Receivable, Recorded Investment
2020	7,561
2019	23,820
2018	17,186
2017	8,093
2016	6,059
Prior	7,697
Revolving Loans	10,473
Total loans held for investment, net	80,889
Loans, gross	80,889	82,926
Consumer | Performing
Financing Receivable, Recorded Investment
2020	7,561
2019	23,785
2018	17,137
2017	8,025
2016	6,031
Prior	7,657
Revolving Loans	10,471
Total loans held for investment, net	80,667
Consumer | Nonperforming
Financing Receivable, Recorded Investment
2020	0
2019	35
2018	49
2017	68
2016	28
Prior	40
Revolving Loans	2
Total loans held for investment, net	$ 222
Consumer | Pass
Financing Receivable, Recorded Investment
Loans, gross		82,650
Consumer | Special mention / watch
Financing Receivable, Recorded Investment
Loans, gross		39
Consumer | Substandard
Financing Receivable, Recorded Investment
Loans, gross		218
Consumer | Doubtful
Financing Receivable, Recorded Investment
Loans, gross		19
Consumer | Loss
Financing Receivable, Recorded Investment
Loans, gross		$ 0