Segment Information (Details) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2020 USD ($)	Sep. 30, 2019 USD ($)	Sep. 30, 2020 USD ($) Segment	Sep. 30, 2019 USD ($)	Dec. 31, 2019 USD ($)
Segment Information [Abstract]
Number of business units | Segment			2
Assets, Revenue and Income (loss) before income taxes for each business unit [Abstract]
Assets	$ 384,952		$ 384,952		$ 377,626
Revenue	46,269	$ 47,745	135,416	$ 145,890
Income (loss) before income taxes	2,433	(1,744)	487	(2,048)
Operating Segments [Member] | American Southern [Member]
Assets, Revenue and Income (loss) before income taxes for each business unit [Abstract]
Assets	151,865		151,865		141,524
Revenue	15,649	14,605	47,772	45,580
Income (loss) before income taxes	1,857	471	4,822	3,849
Operating Segments [Member] | Bankers Fidelity [Member]
Assets, Revenue and Income (loss) before income taxes for each business unit [Abstract]
Assets	221,982		221,982		224,122
Revenue	30,875	32,311	88,619	97,931
Income (loss) before income taxes	2,765	(1,334)	2,023	(3,828)
Corporate and Other [Member]
Assets, Revenue and Income (loss) before income taxes for each business unit [Abstract]
Assets	11,105		11,105		$ 11,980
Revenue	(255)	829	(975)	2,379
Income (loss) before income taxes	$ (2,189)	$ (881)	$ (6,358)	$ (2,069)