Segment Information (Details) $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2015 USD ($)	Jun. 30, 2014 USD ($)	Jun. 30, 2015 USD ($) Segment	Jun. 30, 2014 USD ($)
Segment Information [Abstract]
Number of business units | Segment			2
Revenue and income before income taxes for each business unit [Abstract]
Total revenue	$ 44,772	$ 41,586	$ 85,721	$ 82,746
Income before income taxes	4,189	986	5,199	1,979
Corporate and Other [Member]
Revenue and income before income taxes for each business unit [Abstract]
Total revenue	288	146	395	267
Income before income taxes	(1,535)	(1,286)	(3,564)	(2,998)
American Southern [Member] | Operating Segments [Member]
Revenue and income before income taxes for each business unit [Abstract]
Total revenue	15,664	14,147	30,436	28,348
Income before income taxes	1,824	1,138	3,673	2,195
Bankers Fidelity [Member] | Operating Segments [Member]
Revenue and income before income taxes for each business unit [Abstract]
Total revenue	28,820	27,293	54,890	54,131
Income before income taxes	$ 3,900	$ 1,134	$ 5,090	$ 2,782