Commitments, Contingencies and Related Party Transactions (Guarantees Outstanding) (Details) - Loans sold or securitized with recourse - Loans Sold or Securitized with Recourse - USD ($) $ in Thousands	Dec. 31, 2019	Dec. 31, 2018
Single Family Residential and Multi-Family Residential with recourse
Guarantees
Maximum potential future payments	$ 28,470	$ 33,758
Carrying value	53,286	86,674
Single Family Residential with recourse
Guarantees
Maximum potential future payments	12,578	16,700
Carrying value	12,578	16,700
Multifamily Residential with recourse
Guarantees
Maximum potential future payments	15,892	17,058
Carrying value	$ 40,708	$ 69,974