Segment Information (Details) - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2018	Sep. 30, 2017	Sep. 30, 2018	Sep. 30, 2017	Dec. 31, 2017
Segment Information
Total interest income	$ 37,577	$ 26,006	$ 100,738	$ 67,477
Total interest expense	14,095	7,616	34,942	19,832
Net interest income	23,482	18,390	65,796	47,645
Provision for loan losses	617	592	3,021	1,072
Net Interest Income After Provision for Loan Losses	22,865	17,798	62,775	46,573
Total noninterest income	11,907	8,056	34,850	32,773
Noninterest expense	12,449	8,942	34,719	23,844
Income Before Income Taxes	22,323	16,912	62,906	55,502
Income Tax Expense (Benefit)	5,584	6,445	15,454	21,147
Net Income	16,739	10,467	47,452	34,355
Total assets	3,806,949	3,237,485	3,806,949	3,237,485	$ 3,393,133
Other
Segment Information
Total interest income	700		1,294
Total interest expense	(1,029)	(143)	(2,305)	(350)
Net interest income	1,729	143	3,599	350
Net Interest Income After Provision for Loan Losses	1,729	143	3,599	350
Total noninterest income	(564)		(1,432)
Noninterest expense	2,365	1,610	6,481	3,892
Income Before Income Taxes	(1,200)	(1,467)	(4,314)	(3,542)
Income Tax Expense (Benefit)	(321)	(559)	(1,030)	(1,350)
Net Income	(879)	(908)	(3,284)	(2,192)
Total assets	20,054	(54,505)	20,054	(54,505)
Multifamily | Operating Segments
Segment Information
Total interest income	201	101	472	280
Net interest income	201	101	472	280
Net Interest Income After Provision for Loan Losses	201	101	472	280
Total noninterest income	10,874	6,636	32,000	29,084
Noninterest expense	4,346	2,713	11,946	7,085
Income Before Income Taxes	6,729	4,024	20,526	22,279
Income Tax Expense (Benefit)	1,866	1,534	5,414	8,489
Net Income	4,863	2,490	15,112	13,790
Total assets	152,035	135,530	152,035	135,530
Mortgage Warehousing | Operating Segments
Segment Information
Total interest income	16,335	13,795	42,850	36,307
Total interest expense	7,289	3,657	16,499	9,738
Net interest income	9,046	10,138	26,351	26,569
Provision for loan losses	227	(155)	1,185	250
Net Interest Income After Provision for Loan Losses	8,819	10,293	25,166	26,319
Total noninterest income	779	776	1,949	2,079
Noninterest expense	1,998	2,109	5,919	5,839
Income Before Income Taxes	7,600	8,960	21,196	22,559
Income Tax Expense (Benefit)	1,946	3,414	5,138	8,595
Net Income	5,654	5,546	16,058	13,964
Total assets	1,624,375	1,192,377	1,624,375	1,192,377
Banking | Operating Segments
Segment Information
Total interest income	20,341	12,110	56,122	30,890
Total interest expense	7,835	4,102	20,748	10,444
Net interest income	12,506	8,008	35,374	20,446
Provision for loan losses	390	747	1,836	822
Net Interest Income After Provision for Loan Losses	12,116	7,261	33,538	19,624
Total noninterest income	818	644	2,333	1,610
Noninterest expense	3,740	2,510	10,373	7,028
Income Before Income Taxes	9,194	5,395	25,498	14,206
Income Tax Expense (Benefit)	2,093	2,056	5,932	5,413
Net Income	7,101	3,339	19,566	8,793
Total assets	$ 2,010,485	$ 1,964,083	$ 2,010,485	$ 1,964,083