Goodwill and Intangibles - Acquisitions, Goodwill (Details) - USD ($)				12 Months Ended
	Oct. 01, 2018	Jan. 02, 2018	Aug. 15, 2017	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Goodwill
Goodwill acquired during the period					$ 13,163,000	$ 3,379,000
Post-acquisition adjustments				$ (1,632,000)	412,000
Multifamily
Goodwill
Goodwill acquired during the period						$ 3,379,000
Post-acquisition adjustments					412,000
Banking
Goodwill
Goodwill acquired during the period					8,163,000
Post-acquisition adjustments				(333,000)
Mortgage Warehousing
Goodwill
Goodwill acquired during the period					5,000,000
Post-acquisition adjustments				(1,299,000)
NattyMac, LLC
Goodwill
Post-acquisition adjustments				271,000
Transfer to intangible assets				1,600,000	1,600,000
NattyMac, LLC | Mortgage Warehousing
Goodwill
Goodwill acquired during the period					$ 3,700,000
FMNBP | Banking
Goodwill
Goodwill acquired during the period	$ 6,900,000
Post-acquisition adjustments	$ 333,000
FMBI | Banking
Goodwill
Goodwill acquired during the period		$ 988,000
MCS | Multifamily
Goodwill
Goodwill acquired during the period			$ 3,800,000
Post-acquisition adjustments				$ 412,000