Segment Information (Details) - USD ($) $ in Thousands	3 Months Ended						9 Months Ended
	Sep. 30, 2019	Jun. 30, 2019	Mar. 31, 2019	Sep. 30, 2018	Jun. 30, 2018	Mar. 31, 2018	Sep. 30, 2019	Sep. 30, 2018	Dec. 31, 2018
Segment Information
Total interest income	$ 59,761			$ 37,577			$ 148,196	$ 100,738
Total interest expense	27,137			14,095			63,519	34,942
Net Interest Income	32,624			23,482			84,677	65,796
Provision for loan losses	1,193			617			1,947	3,021
Net Interest Income After Provision for Loan Losses	31,431			22,865			82,730	62,775
Total noninterest income	10,852			11,907			24,386	34,850
Total noninterest expense	15,522			12,449			44,477	34,719
Income Before Income Taxes	26,761			22,323			62,639	62,906
Income taxes	6,502			5,584			15,371	15,454
Net Income	20,259	$ 16,439	$ 10,570	16,739	$ 15,652	$ 15,061	47,268	47,452
Total assets	6,337,186			3,806,949			6,337,186	3,806,949	$ 3,884,163
Other
Segment Information
Total interest income	517			700			1,481	1,294
Total interest expense	(1,758)			(1,029)			(4,691)	(2,305)
Net Interest Income	2,275			1,729			6,172	3,599
Net Interest Income After Provision for Loan Losses	2,275			1,729			6,172	3,599
Total noninterest income	(739)			(564)			(2,090)	(1,432)
Total noninterest expense	2,897			2,365			8,440	6,481
Income Before Income Taxes	(1,361)			(1,200)			(4,358)	(4,314)
Income taxes	(306)			(321)			(1,178)	(1,030)
Net Income	(1,055)			(879)			(3,180)	(3,284)
Total assets	42,323			20,054			42,323	20,054
Multifamily | Operating Segments
Segment Information
Total interest income	335			201			1,000	472
Net Interest Income	335			201			1,000	472
Net Interest Income After Provision for Loan Losses	335			201			1,000	472
Total noninterest income	9,759			10,874			20,868	32,000
Total noninterest expense	6,088			4,346			15,392	11,946
Income Before Income Taxes	4,006			6,729			6,476	20,526
Income taxes	1,265			1,866			1,930	5,414
Net Income	2,741			4,863			4,546	15,112
Total assets	169,283			152,035			169,283	152,035
Mortgage Warehousing | Operating Segments
Segment Information
Total interest income	30,946			16,335			67,276	42,850
Total interest expense	16,157			7,289			35,281	16,499
Net Interest Income	14,789			9,046			31,995	26,351
Provision for loan losses	415			227			875	1,185
Net Interest Income After Provision for Loan Losses	14,374			8,819			31,120	25,166
Total noninterest income	2,699			779			4,593	1,949
Total noninterest expense	2,829			1,998			7,955	5,919
Income Before Income Taxes	14,244			7,600			27,758	21,196
Income taxes	3,320			1,946			6,682	5,138
Net Income	10,924			5,654			21,076	16,058
Total assets	3,529,267			1,624,375			3,529,267	1,624,375
Banking | Operating Segments
Segment Information
Total interest income	27,963			20,341			78,439	56,122
Total interest expense	12,738			7,835			32,929	20,748
Net Interest Income	15,225			12,506			45,510	35,374
Provision for loan losses	778			390			1,072	1,836
Net Interest Income After Provision for Loan Losses	14,447			12,116			44,438	33,538
Total noninterest income	(867)			818			1,015	2,333
Total noninterest expense	3,708			3,740			12,690	10,373
Income Before Income Taxes	9,872			9,194			32,763	25,498
Income taxes	2,223			2,093			7,937	5,932
Net Income	7,649			7,101			24,826	19,566
Total assets	$ 2,596,313			$ 2,010,485			$ 2,596,313	$ 2,010,485