Office Properties and Equipment, net - Schedule of Office Properties and Equipment (Details) - USD ($) $ in Thousands	Dec. 31, 2018	Dec. 31, 2017
Property, Plant and Equipment [Line Items]
Office property and equipment, gross	$ 106,528	$ 93,287
Less accumulated depreciation and amortization	54,478	50,667
Total office properties and equipment, net	52,050	42,620
Land
Property, Plant and Equipment [Line Items]
Office property and equipment, gross	7,829	7,829
Buildings
Property, Plant and Equipment [Line Items]
Office property and equipment, gross	24,018	24,018
Land and building improvements
Property, Plant and Equipment [Line Items]
Office property and equipment, gross	24,864	15,583
Leasehold improvements
Property, Plant and Equipment [Line Items]
Office property and equipment, gross	21,279	19,821
Furniture and equipment
Property, Plant and Equipment [Line Items]
Office property and equipment, gross	$ 28,538	$ 26,036