Employee Benefit and Deferred Compensation Plans (Details Textual) (USD $) In Thousands, except Share data, unless otherwise specified	1 Months Ended	3 Months Ended			6 Months Ended
	Jan. 31, 2013	Jun. 30, 2014	Sep. 30, 2013	Jun. 30, 2013	Jun. 30, 2014	Jun. 30, 2013	Dec. 31, 2013
Share-based Compensation Arrangement by Share-based Payment Award [Line Items]
Share based compensation arrangement by share based payment stock option granted, vested and exercisable (percent)	33.33%
Award vesting period	3 years
Option increases (decreases) in period (shares)			11,557		0
Weighted average exercise price (usd per share)			$ 21.16		$ 0.00
Weighted average remaining contractual life of options assumed in acquisition			2 years 18 days
Fair value of options assumed in acquisition			$ 68
Options outstanding (shares)		915,245			915,245		1,060,350
Weighted average exercise price (usd per share)		$ 18.90			$ 18.90		$ 18.64
Treasury shares reissued (shares)					131,973
Total stock-based compensation expense		$ 951		$ 477	$ 1,822	$ 955
First M&F | Incentive Plan and Stock Option Plan
Share-based Compensation Arrangement by Share-based Payment Award [Line Items]
Weighted average remaining contractual life of options assumed in acquisition					1 year 2 months 27 days
Options outstanding (shares)		6,934			6,934
Weighted average exercise price (usd per share)		$ 24.61			$ 24.61