Segment Information (Details) (USD $) In Thousands, unless otherwise specified	3 Months Ended		9 Months Ended
	Sep. 30, 2014	Sep. 30, 2013	Sep. 30, 2014 Segment	Sep. 30, 2013
Segment Information [Abstract]
Number of business units			2
Revenue and income before income taxes for each business unit [Abstract]
Total revenue	$ 42,656	$ 43,247	$ 125,402	$ 124,545
Income before income taxes	1,587	2,689	3,566	9,763
Corporate and Other [Member]
Revenue and income before income taxes for each business unit [Abstract]
Total revenue	928	217	1,195	1,217
Income before income taxes	(964)	(1,668)	(3,962)	(3,937)
American Southern [Member] | Operating Segments [Member]
Revenue and income before income taxes for each business unit [Abstract]
Total revenue	14,819	14,994	43,167	40,231
Income before income taxes	1,500	1,912	3,695	6,531
Bankers Fidelity [Member] | Operating Segments [Member]
Revenue and income before income taxes for each business unit [Abstract]
Total revenue	26,909	28,036	81,040	83,097
Income before income taxes	$ 1,051	$ 2,445	$ 3,833	$ 7,169