Loans and Allowance for Loan Losses - Credit Risk Profile of Loan Portfolio (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Credit risk profile of portfolio
Loans	$ 3,520,653	$ 3,028,310
Special Mention (Watch)
Credit risk profile of portfolio
Loans	48,340	60,305
Substandard
Credit risk profile of portfolio
Loans	14,124	12,517
Acceptable and Above
Credit risk profile of portfolio
Loans	3,458,189	2,955,488
Mortgage warehouse lines of credit
Credit risk profile of portfolio
Loans	1,083,776	765,151
Mortgage warehouse lines of credit | Substandard
Credit risk profile of portfolio
Loans	230	233
Mortgage warehouse lines of credit | Acceptable and Above
Credit risk profile of portfolio
Loans	1,083,546	764,918
Residential real estate
Credit risk profile of portfolio
Loans	421,978	413,835
Residential real estate | Special Mention (Watch)
Credit risk profile of portfolio
Loans	2,413	2,472
Residential real estate | Substandard
Credit risk profile of portfolio
Loans	2,410	3,109
Residential real estate | Acceptable and Above
Credit risk profile of portfolio
Loans	417,155	408,254
Multi-family and healthcare financing
Credit risk profile of portfolio
Loans	1,435,206	1,347,125
Multi-family and healthcare financing | Special Mention (Watch)
Credit risk profile of portfolio
Loans	30,930	41,882
Multi-family and healthcare financing | Acceptable and Above
Credit risk profile of portfolio
Loans	1,404,276	1,305,243
Commercial and commercial real estate
Credit risk profile of portfolio
Loans	468,668	398,601
Commercial and commercial real estate | Special Mention (Watch)
Credit risk profile of portfolio
Loans	12,961	13,806
Commercial and commercial real estate | Substandard
Credit risk profile of portfolio
Loans	9,425	9,152
Commercial and commercial real estate | Acceptable and Above
Credit risk profile of portfolio
Loans	446,282	375,643
Agricultural production and real estate
Credit risk profile of portfolio
Loans	92,498	85,210
Agricultural production and real estate | Special Mention (Watch)
Credit risk profile of portfolio
Loans	2,006	2,114
Agricultural production and real estate | Substandard
Credit risk profile of portfolio
Loans	2,043
Agricultural production and real estate | Acceptable and Above
Credit risk profile of portfolio
Loans	88,449	83,096
Consumer and margin loans
Credit risk profile of portfolio
Loans	18,527	18,388
Consumer and margin loans | Special Mention (Watch)
Credit risk profile of portfolio
Loans	30	31
Consumer and margin loans | Substandard
Credit risk profile of portfolio
Loans	16	23
Consumer and margin loans | Acceptable and Above
Credit risk profile of portfolio
Loans	$ 18,481	$ 18,334