Segment Information (Details) - USD ($) $ in Thousands	3 Months Ended								12 Months Ended
	Dec. 31, 2017	Sep. 30, 2017	Jun. 30, 2017	Mar. 31, 2017	Dec. 31, 2016	Sep. 30, 2016	Jun. 30, 2016	Mar. 31, 2016	Dec. 31, 2017	Dec. 31, 2016	Dec. 31, 2015
Segment Information
Total interest income	$ 26,910	$ 26,006	$ 22,464	$ 19,007	$ 20,691	$ 19,827	$ 16,929	$ 15,492	$ 94,387	$ 72,939	$ 56,345
Total interest expense	7,958	7,616	6,740	5,476	5,509	4,857	4,360	4,242	27,790	18,968	14,290
Net interest income	18,952	18,390	15,724	13,531	15,182	14,970	12,569	11,250	66,597	53,971	42,055
Provision for loan losses	1,400	592	240	240	240	240	240	240	2,472	960	960
Net Interest Income After Provision for Loan Losses	17,552	17,798	15,484	13,291	14,942	14,730	12,329	11,010	64,125	53,011	41,095
Total noninterest income	14,907	8,056	16,626	8,091	6,924	11,629	6,658	3,293	47,680	28,504	27,008
Noninterest expense	10,800	8,942	8,261	6,641	7,384	7,173	6,376	5,787	34,644	26,720	20,922
Income Before Income Taxes	21,659	16,912	23,849	14,741	14,482	19,186	12,611	8,516	77,161	54,795	47,181
Income Tax Expense (Benefit)	1,330	6,445	9,091	5,611	5,728	7,587	4,986	3,367	22,477	21,668	18,798
Net Income	20,329	$ 10,467	$ 14,758	$ 9,130	8,754	$ 11,599	$ 7,625	$ 5,149	54,684	33,127	28,383
Total assets	3,393,133				2,718,512				3,393,133	2,718,512	2,269,442
Other
Segment Information
Total interest income									721
Total interest expense									(736)	1,187	1,193
Net interest income									1,457	(1,187)	(1,193)
Net Interest Income After Provision for Loan Losses									1,457	(1,187)	(1,193)
Total noninterest income									(814)
Noninterest expense									6,188	3,133	1,894
Income Before Income Taxes									(5,545)	(4,320)	(3,087)
Income Tax Expense (Benefit)									(1,917)	(1,708)	(1,230)
Net Income									(3,628)	(2,612)	(1,857)
Total assets	16,855				13,453				16,855	13,453	4,121
Multifamily | Operating Segments
Segment Information
Total interest income									414	237	309
Net interest income									414	237	309
Net Interest Income After Provision for Loan Losses									414	237	309
Total noninterest income									43,715	22,798	24,749
Noninterest expense									10,911	7,474	6,312
Income Before Income Taxes									33,218	15,561	18,746
Income Tax Expense (Benefit)									4,557	6,153	7,469
Net Income									28,661	9,408	11,277
Total assets	134,390				98,553				134,390	98,553	87,188
Mortgage Warehousing | Operating Segments
Segment Information
Total interest income									48,798	43,352	36,830
Total interest expense									13,673	11,235	9,606
Net interest income									35,125	32,117	27,224
Provision for loan losses									452	(247)	(47)
Net Interest Income After Provision for Loan Losses									34,673	32,364	27,271
Total noninterest income									2,836	3,220	1,193
Noninterest expense									7,710	7,730	5,703
Income Before Income Taxes									29,799	27,854	22,761
Income Tax Expense (Benefit)									11,558	11,015	9,068
Net Income									18,241	16,839	13,693
Total assets	1,352,748				1,060,723				1,352,748	1,060,723	924,731
Banking | Operating Segments
Segment Information
Total interest income									44,454	29,350	19,206
Total interest expense									14,853	6,546	3,491
Net interest income									29,601	22,804	15,715
Provision for loan losses									2,020	1,207	1,007
Net Interest Income After Provision for Loan Losses									27,581	21,597	14,708
Total noninterest income									1,943	2,486	1,066
Noninterest expense									9,835	8,383	7,013
Income Before Income Taxes									19,689	15,700	8,761
Income Tax Expense (Benefit)									8,279	6,208	3,491
Net Income									11,410	9,492	5,270
Total assets	$ 1,889,140				$ 1,545,783				$ 1,889,140	$ 1,545,783	$ 1,253,402