Investment Securities, Securities Available-for-Sale (Details) - USD ($) $ in Thousands		Sep. 30, 2020	Dec. 31, 2019	Sep. 30, 2019
Amortized cost, fair values, and unrealized gains and losses of securities available-for-sale [Abstract]
Amortized cost		$ 550,053	$ 502,693	$ 488,503
Gross unrealized gains		18,722	4,905	4,710
Gross unrealized losses		239	212	283
Fair/Book value		568,536	507,386	492,930
US Treasury Notes [Member]
Amortized cost, fair values, and unrealized gains and losses of securities available-for-sale [Abstract]
Amortized cost		94,830	54,745	129,724
Gross unrealized gains		512	250	270
Gross unrealized losses		0	0	5
Fair/Book value		95,342	54,995	129,989
US Government Agency SBA [Member]
Amortized cost, fair values, and unrealized gains and losses of securities available-for-sale [Abstract]
Amortized cost		8,530	10,902	11,844
Gross unrealized gains		1	9	15
Gross unrealized losses		102	113	115
Fair/Book value		8,429	10,798	11,744
Mortgage Backed Securities [Member]
Amortized cost, fair values, and unrealized gains and losses of securities available-for-sale [Abstract]
Amortized cost	[1]	421,263	436,531	343,295
Gross unrealized gains	[1]	18,050	4,646	4,425
Gross unrealized losses	[1]	39	99	163
Fair/Book value	[1]	439,274	441,078	347,557
Corporate Securities [Member]
Amortized cost, fair values, and unrealized gains and losses of securities available-for-sale [Abstract]
Amortized cost		25,002
Gross unrealized gains		159
Gross unrealized losses		98
Fair/Book value		25,063
Other [Member]
Amortized cost, fair values, and unrealized gains and losses of securities available-for-sale [Abstract]
Amortized cost		428	515	3,640
Gross unrealized gains		0	0	0
Gross unrealized losses		0	0	0
Fair/Book value		$ 428	$ 515	$ 3,640

[1] All Mortgage Backed Securities consist of securities collateralized by residential real estate and were issued by an agency or government sponsored entity of the U.S. government.