Segment Information (Details) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2016 USD ($)	Sep. 30, 2015 USD ($)	Sep. 30, 2016 USD ($) Segment	Sep. 30, 2015 USD ($)
Segment Information [Abstract]
Number of business units | Segment			2
Revenue and income before income taxes for each business unit [Abstract]
Total revenue	$ 42,447	$ 40,359	$ 126,048	$ 126,080
Income before income taxes	317	365	2,050	5,564
Corporate and Other [Member]
Revenue and income before income taxes for each business unit [Abstract]
Total revenue	110	90	358	485
Income before income taxes	(1,702)	(1,497)	(5,032)	(5,061)
American Southern [Member] | Operating Segments [Member]
Revenue and income before income taxes for each business unit [Abstract]
Total revenue	14,112	14,899	43,743	45,335
Income before income taxes	1,855	1,572	5,971	5,245
Bankers Fidelity [Member] | Operating Segments [Member]
Revenue and income before income taxes for each business unit [Abstract]
Total revenue	28,225	25,370	81,947	80,260
Income before income taxes	$ 164	$ 290	$ 1,111	$ 5,380