Fair Value of Financial Instruments - Schedule of Fair Value of Financial Instruments (Details) - USD ($) $ in Thousands	Mar. 31, 2019	Dec. 31, 2018
ASSETS:
Cash and cash equivalents	$ 105,258	$ 109,556
Investment securities available-for-sale (at fair value)	749,537	791,102
Mortgage-backed securities-residential mortgage pass-through securities issued or guaranteed by U.S. Government agencies or sponsored enterprises held-to-maturity	221,727	235,398
Non-marketable securities	24,574	27,555
Loans held-for-sale	59,324	48,120
LIABILITIES:
Time deposits	1,081,092	1,080,529
Securities sold under agreements to repurchase	59,543	66,047
Federal Home Loan Bank advances	228,421	301,660
Residential Mortgage Pass-Through Securities Issued or Guaranteed by U.S. Government Agencies or Sponsored Enterprises [Member]
ASSETS:
Investment securities available-for-sale (at fair value)	134,488	146,642
Mortgage-backed securities-residential mortgage pass-through securities issued or guaranteed by U.S. Government agencies or sponsored enterprises held-to-maturity	147,221	157,115
Other Residential MBS Issued or Guaranteed by U.S. Government Agencies or Sponsored Enterprises [Member]
ASSETS:
Investment securities available-for-sale (at fair value)	613,970	643,381
Mortgage-backed securities-residential mortgage pass-through securities issued or guaranteed by U.S. Government agencies or sponsored enterprises held-to-maturity	74,506	78,283
Other Securities [Member]
ASSETS:
Investment securities available-for-sale (at fair value)	469	469
Municipal [Member]
ASSETS:
Investment securities available-for-sale (at fair value)	610	610
Carrying Amount [Member] | Level 1 [Member]
ASSETS:
Cash and cash equivalents	105,258	109,556
Carrying Amount [Member] | Level 2 [Member]
ASSETS:
Non-marketable securities	24,574	27,555
Loans held-for-sale	59,324	48,120
Accrued interest receivable	22,362	17,852
LIABILITIES:
Deposit transaction accounts	3,633,356	3,445,092
Time deposits	1,081,092	1,080,529
Securities sold under agreements to repurchase	59,543	66,047
Federal Home Loan Bank advances	228,421	301,660
Accrued interest payable	8,731	6,889
Carrying Amount [Member] | Level 3 [Member]
ASSETS:
Loans receivable, net	4,246,941	4,092,308
Carrying Amount [Member] | Residential Mortgage Pass-Through Securities Issued or Guaranteed by U.S. Government Agencies or Sponsored Enterprises [Member] | Level 2 [Member]
ASSETS:
Investment securities available-for-sale (at fair value)	134,488	146,642
Mortgage-backed securities-residential mortgage pass-through securities issued or guaranteed by U.S. Government agencies or sponsored enterprises held-to-maturity	147,221	157,115
Carrying Amount [Member] | Other Residential MBS Issued or Guaranteed by U.S. Government Agencies or Sponsored Enterprises [Member] | Level 2 [Member]
ASSETS:
Investment securities available-for-sale (at fair value)	613,970	643,381
Mortgage-backed securities-residential mortgage pass-through securities issued or guaranteed by U.S. Government agencies or sponsored enterprises held-to-maturity	74,506	78,283
Carrying Amount [Member] | Other Securities [Member] | Level 3 [Member]
ASSETS:
Investment securities available-for-sale (at fair value)	469	469
Carrying Amount [Member] | Municipal [Member] | Level 2 [Member]
ASSETS:
Municipal obligations	441	441
Carrying Amount [Member] | Municipal [Member] | Level 3 [Member]
ASSETS:
Municipal obligations	169	169
Estimated Fair Value [Member] | Level 1 [Member]
ASSETS:
Cash and cash equivalents	105,258	109,556
Estimated Fair Value [Member] | Level 2 [Member]
ASSETS:
Non-marketable securities	24,574	27,555
Loans held-for-sale	59,324	48,120
Accrued interest receivable	22,362	17,852
LIABILITIES:
Deposit transaction accounts	3,633,356	3,445,092
Time deposits	1,072,986	1,068,233
Securities sold under agreements to repurchase	59,543	66,047
Federal Home Loan Bank advances	228,810	301,933
Accrued interest payable	8,731	6,889
Estimated Fair Value [Member] | Level 3 [Member]
ASSETS:
Loans receivable, net	4,279,916	4,082,146
Estimated Fair Value [Member] | Residential Mortgage Pass-Through Securities Issued or Guaranteed by U.S. Government Agencies or Sponsored Enterprises [Member] | Level 2 [Member]
ASSETS:
Investment securities available-for-sale (at fair value)	134,488	146,642
Mortgage-backed securities-residential mortgage pass-through securities issued or guaranteed by U.S. Government agencies or sponsored enterprises held-to-maturity	146,316	154,412
Estimated Fair Value [Member] | Other Residential MBS Issued or Guaranteed by U.S. Government Agencies or Sponsored Enterprises [Member] | Level 2 [Member]
ASSETS:
Investment securities available-for-sale (at fair value)	613,970	643,381
Mortgage-backed securities-residential mortgage pass-through securities issued or guaranteed by U.S. Government agencies or sponsored enterprises held-to-maturity	73,291	76,514
Estimated Fair Value [Member] | Other Securities [Member] | Level 3 [Member]
ASSETS:
Investment securities available-for-sale (at fair value)	469	469
Estimated Fair Value [Member] | Municipal [Member] | Level 2 [Member]
ASSETS:
Municipal obligations	441	441
Estimated Fair Value [Member] | Municipal [Member] | Level 3 [Member]
ASSETS:
Municipal obligations	169	169
Interest rate swap derivatives [Member] | Carrying Amount [Member] | Level 2 [Member]
ASSETS:
Derivative asset	14,035	20,627
LIABILITIES:
Derivative liability	7,772	3,577
Interest rate swap derivatives [Member] | Estimated Fair Value [Member] | Level 2 [Member]
ASSETS:
Derivative asset	14,035	20,627
LIABILITIES:
Derivative liability	7,772	3,577
Mortgage banking derivatives | Carrying Amount [Member] | Level 3 [Member]
ASSETS:
Derivative asset	1,552	871
LIABILITIES:
Derivative liability	811	544
Mortgage banking derivatives | Estimated Fair Value [Member] | Level 3 [Member]
ASSETS:
Derivative asset	1,552	871
LIABILITIES:
Derivative liability	$ 811	$ 544