The surprise was partly because the airline was firmly in the black by 1997 recording profits of around $900 million for the year which itself followed a record $662 million profit in 1996, whilst its stock was up 17% within a year. In retrospect however Allen's departure seems less of a surprise.
"My personal goals during the past three years were to lead Delta Air Lines back to sustained profitability and position our company to be poised for growth to make Delta a leader. Delta has realized great progress in both of these areas, and the company is in excellent condition."
If this was true it wasn't always so. Allen's tenure had seen Delta through (or made worse depending on your viewpoint) a period of major strife.
In the end though it appears the reason for his departure was financial. With Delta's stock over $100 he asked for a massive 10 year contract full of stock incentives which the airline balked at. That didn't stop him making a fortune more from the airline however, as he gained an insane severance package worth $500,000 a year for 8 years plus other frills which amounted to tens of millions more.
The Inquirer, 1994. Delta Planning Up To 15,000 Job Cuts To Slice Costs The Chairman Could Be On His Way Out. The Airline Has Lost $1.9 Billion Since 1991.
Boulton G, 1997. Delta chief's resignation a surprise - Cincinnati Enquirer
Bynum, R. 1997. Delta chief announces sudden retirement at 55. Associated Press
1997 Blomberg, WHY CEO ALLEN IS LEAVING DELTA AIRLINES
Institute of Management and Administration, Cost Reduction and Control Best Practices
FlightGlobal, 2000. Delta continues rebranding with another livery revamp
Grantham, R, 2005. Millions richer, ex-CEO departs Delta payroll. Atlanta Journal.