Loans - Recorded Investment in Residential, Consumer and Credit Card Loans Based on Payment Activity (Details) - USD ($) $ in Thousands	Dec. 31, 2020	Jan. 01, 2020	Dec. 31, 2019
Financing Receivable, Credit Quality Indicator [Line Items]
Amount of recorded investment	$ 3,337,449	$ 2,804,035	$ 2,784,562
Credit Card, Consumer
Financing Receivable, Credit Quality Indicator [Line Items]
Amount of recorded investment	8,115
Residential Real Estate
Financing Receivable, Credit Quality Indicator [Line Items]
Amount of recorded investment		814,030	814,030
Consumer
Financing Receivable, Credit Quality Indicator [Line Items]
Amount of recorded investment			124,785
Credit Cards
Financing Receivable, Credit Quality Indicator [Line Items]
Amount of recorded investment		$ 7,569	7,569
Performing | Credit Card, Consumer
Financing Receivable, Credit Quality Indicator [Line Items]
Amount of recorded investment	8,081
Performing | Residential Real Estate
Financing Receivable, Credit Quality Indicator [Line Items]
Amount of recorded investment			809,247
Performing | Consumer
Financing Receivable, Credit Quality Indicator [Line Items]
Amount of recorded investment			123,950
Performing | Credit Cards
Financing Receivable, Credit Quality Indicator [Line Items]
Amount of recorded investment			7,567
Nonperforming | Credit Card, Consumer
Financing Receivable, Credit Quality Indicator [Line Items]
Amount of recorded investment	$ 34
Nonperforming | Residential Real Estate
Financing Receivable, Credit Quality Indicator [Line Items]
Amount of recorded investment			4,783
Nonperforming | Consumer
Financing Receivable, Credit Quality Indicator [Line Items]
Amount of recorded investment			835
Nonperforming | Credit Cards
Financing Receivable, Credit Quality Indicator [Line Items]
Amount of recorded investment			$ 2