You may be charged loan origination fees and quarterly fees When you borrow against the retirement account balance, you sign a loan agreement that spells out the principal, the term of the loan, the interest rate, any fees, and other terms that may apply.