Loan Servicing (Details) - USD ($) $ in Thousands	Dec. 31, 2020	Dec. 31, 2019	Dec. 31, 2018
Certain Loans Acquired in Transfer Not Accounted for as Debt Securities Acquired During Period [Line Items]
Unpaid balance of loans serviced for others	$ 635,224	$ 514,294	$ 533,386
Federal Home Loan Mortgage Corporation
Certain Loans Acquired in Transfer Not Accounted for as Debt Securities Acquired During Period [Line Items]
Unpaid balance of loans serviced for others	514,539	354,393	351,672
Fannie Mae
Certain Loans Acquired in Transfer Not Accounted for as Debt Securities Acquired During Period [Line Items]
Unpaid balance of loans serviced for others	69,072	82,922	92,552
Federal Home Loan Bank
Certain Loans Acquired in Transfer Not Accounted for as Debt Securities Acquired During Period [Line Items]
Unpaid balance of loans serviced for others	51,479	76,815	88,931
Chevy Chase Mortgage Company
Certain Loans Acquired in Transfer Not Accounted for as Debt Securities Acquired During Period [Line Items]
Unpaid balance of loans serviced for others	$ 134	$ 164	$ 231