Vol. I
p86 Chapter V

Sorrow and Scandal Come to the Lees

In holiday seasons, Robert and Smith Lee sometimes arranged to have their furloughs run simultaneously, and they would join their elder brother, Carter, in mirthful journeyings to the homes of their kinspeople. There would be laughter, teasing, sprightly anecdote, and much harmless gallantry. When liquor was passed, Robert and Smith would decline but Carter was agreeable. "I have always told these boys," he would say, "that I would drink their share of wine, provided they would keep me generously supplied." Carter was the centre of amusement at these parties, for his social gifts were of the highest and his humor keener than that of either of his brothers.​1 Robert's most favorable impression was made by the dignity of his fine person and by his gracious, considerate manners.

On one of his visits, his manners and his regard for his elders brought him no little embarrassment. In the company there chanced to be a bibulous old gentleman who was much pleased with the clean, high-minded cadet. The night before Robert left, this worthy came to the boy's room. To quote a feminine biographer who did not fail to point the moral, the veteran of many a drinking-bout "lamented the idle and useless life into which he had fallen, excusing himself upon the score of loneliness, and the sorrow which weighed upon him in the loss of those most dear. In the most impressive manner he besought his young guest to be warned by his example; prayed him to cherish the good habits he had already acquired, and promised to listen to his entreaties that he would change his own life, and thereby secure more entirely his respect and affection."​2 So runs the story; sober history suggests that the gentleman had been to the shrine of Bacchus before he staggered to the confessional. It probably  p87  took all of Robert's tact of the dispose of the penitent and to get him out of the room.

SYDNEYº SMITH LEE, BROTHER OF ROBERT EDWARD LEE, IN THE DRESS UNIFORM OF A JUNIOR OFFICER OF THE NAVY, FROM A PORTRAIT BY AN UNKNOWN ARTIST Smith Lee, who was four years older than Robert, was regarded by many as the handsomer of the two.

Barring a repetition of such a scene, the summer after his graduation should have been for Robert the happiest of all these seasons of care-free visiting, but it was, instead, one of the saddest periods of his life. The joy of home-coming was ruined by the illness of his mother. Her health had been bad during the winter, though there had been some signs of improvement with the spring.​3 When Robert arrived, she was at Ravensworth in a worse condition than ever, and was ready to die. Charles Carter was developing his practice; Smith was progressing in the navy; Ann in 1826​4 had married William Louis Marshall, a minister who later became an attorney of station; Mildred, in her nineteenth year, was in friendly hands. And now Robert was embarking on a career of high promise. Ann Carter Lee had seen it through, but the struggle had cost her all her vitality. She could fight for nothing further.

Robert immediately resumed his old duties as a nurse. He mixed her medicines, administered them, and watched by her bed almost continuously. When he left her room, her gaze followed him, and she would look steadily at the door until he entered again.​5 It was not a long siege this time. On July 10, Robert saw the light leave her eye and the last faint breath fail her. He turned from the bed in a grief that he never forgot.​6 She was buried at Ravensworth, and there her ashes remained until they were moved to rest in a vault at Lexington, Va., near those of her son, whither also, in 1913,​7 the bones of Henry Lee were brought from Cumberland Island.

Ann Carter Lee was fifty-six when she died. She had been a mother for thirty-one years and a widow for eleven. Nearly all her married life of two decades and a half had been clouded with financial worry. For at least seventeen years before her death, she had carried the burden of maintaining the family on her personal income. Of all that she thought and planned and suffered  p88  during those years, hardly an echo has survived the indifferent roar of a hurrying century. The scant score of letters in her autograph that now remain and the few references to her in the extant correspondence of her kinspeople and friends do not suffice to give more than the most shadowy outlines of her personality. She had patience in misfortune; she used wisely the little that she possessed; she served a God who was very real to her; she kept her friends and she loved her kin; she had the wisdom and skill with which to vitalize for her children the virtue of self-control; she made their interests her own; she must have had much of the Carter interest in life and some of the Carter sense of humor; she had high, uncomplaining courage in facing continued adversity. This much is known. But in what manner she dealt with a spendthrift husband seventeen years her senior, and what she thought of the life she had left at Shirley or of the life she had led among the Lees, and how she went about the rearing of Robert, and whether she believed he would become a great man, she does not tell us. None of her letters to Robert is known to be in existence, and only two of those to Smith are left. The earlier of these exhibits her devotion to her children and to her kinspeople. It follows:

The second letter was written not long before her end, and it showed how the long fight wore on her nerves. It reads:

These letters are the only picture of herself that she has left. Much that a curious world would like to know about the mother of her son can never be established.10

 p92  For a time after her death, Robert apparently was in Georgetown,​11 engaged no doubt in helping to settle his mother's estate. She had prepared a will not long before her death, so there was little difficulty in executing her wishes. To Ann Marshall, she left her maid and the Negress's child, together with three slaves that were then with Mrs. Marshall. To her also went the white tea china, the wardrobe, two of the mother's tablecloths — she had but four — and half the napkins and wearing apparel. Mildred received the old family servant Nat, the carriage and horses, the piano, the other two good tablecloths, and an equal share of the napkins and wearing apparel. Each daughter was given, in addition, $10,000 of the principal of the trust-fund which had been prudently invested in bank stock. The rest of her property Mrs. Lee directed her executor to sell, and to divide the proceeds among her three sons. The size of the bond given when the will was probated in Fairfax County would indicate that the amount allotted the boys was hardly more than $3000 each.12

It was sad, sad business breaking up the home in Georgetown and dividing the treasures to which Ann Lee had clung through her darkest days. Robert doubtless was relieved when he was able to return to Virginia about August 1 and sojourn with relatives. But, with the buoyancy of youth, he quickly recovered from the immediate grief of his mother's death and, as one of his cousins remembered, was "as full of life, fun and particularly teasing, as any of us."​13 He visited much at Eastern View, the Randolph home in Fauquier, but there was another mansion to which his interest and his horse were turning very frequently. This was Arlington, the home of George Washington Parke Custis, on the hills above Alexandria, over­looking the country's capital. Custis was the grandson of Mrs. George Washington and was the adopted son of Washington. Having resided at Mount Vernon from 1782, when he was an infant, until the end of his grandmother's  p93  life in 1802, he had observed Washington closely during the general's last years. His temperament was such that he delighted in the sentimental appellation, "The child of Mount Vernon," which clung to him all his days, though he measured out his full seventy years and more.​14 Arlington had been built by him after the death of his grandmother, when Mount Vernon had reverted to the Washington family. The house, which was named after an old Custis home on the Eastern Shore of Virginia, was distinguished more for its site and for the impressive columnated portico, with Doric capitals, than for interior beauty or convenience. Its rooms, though large, were few and gloomy; the heavy columns dwarfed the mansion. It gives the impression of being built to be looked at, rather than to be lived in.

To Arlington, in 1806, Custis brought as his bride Mary Lee Fitzhugh, daughter of Colonel and Mrs. William Fitzhugh of Chatham, opposite Fredericksburg on the Rappahannock. She was eighteen at the time and he was twenty-five, and they had ahead of them forty-seven years of married life. Of their four children, only one survived infancy. She was a girl, Mary Anne Randolph Custis, born October 1, 1808, and reared in the amplest luxury. Twenty-one years of age when Robert came home from West Point, she had known him almost all her life, for the families were distantly related through the Lee ancestry of the Randolphs and they visited one another frequently.​15 She was something of a toast in the Lee family, as much admired by Robert's brother Carter, as by the boys nearer her own age. "I heard of dear Miss Custis yesterday," Carter had written not long before Robert's return from New York, "and that she was much afflicted with a cold."​16 She was a frail, blonde girl. Her features were aristocratic but they were not beauti­ful. The nose was a trifle too long and the chin a bit too sharp, but she had freshness,  p94  bright eyes, a ready smile, and quick, sympathetic interest. If Robert did not actually love her from boyhood, he certainly put her in a place by herself. She it was who drew him to Arlington. When he went away, it was to come again, always with deepening delight in her company.

While Robert was visiting at Arlington and at Eastern View, one legacy left to Mildred required his care. This was Nat, the Negro coachman and house-servant. Nat was typical of a rather large element among Virginia slaves. He had helped to rear the children; he had served long and loyally; he had shared in all the struggles of the family. None of the Lees ever regarded him otherwise than as a member of the family. And now Nat was sick, with some slow, devitalizing malady. What should be done to provide for him? The carriage would creak no more over the Georgetown streets as Mrs. Lee went out to take the air. The household was broken up. How could he be assured good nursing? While this question was being debated, Robert's orders came. They read as follows:

Cockspur Island! A God-forsaken spot by all accounts, redeemed only by the fact that it was near Savannah, where lived the family of Lee's chum, Jack Mackay. But orders were orders, and besides, as the climate there was mild, Nat's health might be improved. So, when Robert said farewell to all his kinspeople and to the young mistress of Arlington, Nat accompanied him on the long sea-journey to Savannah. It was a curious companion­ship for the beginning of active duty in an army which Lee was to  p95  leave, more than thirty years afterwards in order, his enemies alleged, to fight for the perpetuation of slavery.18

The town at which the young lieutenant and the old Negro arrived by packet, about November 1, 1829, was a place of some 7300 people, the largest city and the principal port in a state that had been settled less than one hundred years and then counted no more than 300,000 whites in a population of 516,000. Savannah had history, for it had been occupied by the British in 1778, and in October of the following year, it had been besieged by the American and French forces. These operations, which were unsuccess­ful, had cost the life of Count Pulaski. With that part of the story of the town, Robert was of course familiar from his father's memoirs, for "Light-Horse Harry" had fought farther up the river on which his son was now to labor as an engineer. The same tide that flooded Savannah, swept Cumberland Island, where Henry Lee had ended his days. Socially, the town was attractive and cultured. The Mackays, who welcomed Lee with open doors, were among the most distinguished of Savannah families, with daughters who were interesting even at first sight. In a few days Lee was introduced to all the civilians who were accounted worth knowing. As for the army, Savannah boasted a small garrison of United States artillery, among whom were several officers with whom Lee became friendly. Jack Mackay had been assigned to this garrison.​19 Another of its officers was Lieutenant James A. Chambers of the Second Artillery, who may have been distantly connected with Lee.20

The post was by no means so pleasant as the town. It was, in fact, as drab and desolate as its reputation. Cockspur Island lies •twelve miles down-stream from Savannah and is the easternmost islet of a number of flats in Tybee Roads, as the mouth of the river is styled. The island is •about a mile in length and about two-thirds as wide. Very little of it was above normal tide level at the time of Lee's arrival, and most of it was marsh-land, flooded daily and completely covered in heavy storms. Up the river was a string of similar swampy islands. Northward, across •nearly two miles of water, were Turtle Island and a tangle of flats on the  p96  mainland, broken by winding estuaries and untouched by man's labor. To the south of Cockspur Island and separated from it by a narrow channel were other swamps, even more confusing and inhospitable. Eastward the open sea was spread. In summer, Cockspur had virtually to be abandoned because of mosquitoes, heat, and fever. It was, however, a training-school for Lee in the practical problems of military engineering and in the management of labor. He came to his duty there at a most advantageous period. Congress had recently begun its first extensive programme of coast defenses, which the engineers had the satisfaction of locating, designing, and constructing. Promotion was very slow, and the jealousy of some high functionaries was pronounced; but there probably was never a time of peace in the history of the corps when it held out so many opportunities, or gave young officers so much responsibility, as it did when Lee joined it.

Lee's orders had indicated that his commanding officer at Cockspur Island was to be Major Samuel Babcock of Massachusetts, one of the earliest graduates of the United States Military Academy. Babcock had been in the army more than twenty years when Robert was graduated and his health was becoming impaired by his exertions. For that reason, the load of his youthful subordinate was heavy from the very outset. Aside from the engineering duties, Lee had to discharge those of acting assistant commissary of subsistence. It was the only time in his life that he labored in that most thankless of army services.21

His engineering work was not always interesting but it usually was troublesome. The project at Cockspur Island was to locate and subsequently to construct a heavy fort on an island that afforded at best, a doubtful foundation.​22 After the site was chosen, embankments had to be reared to keep out the tide. Then a canal had to be constructed, and when this had drained the site, the fort was to be laid out. Into the first stages of this hard work, Robert put all he had learned at West Point and all the strength of his staunch physique. He spent so many days in mud and water, up to his arm-pits, that a certain interested young woman,  p97  up in Virginia, wondered how he ever survived it, and to the end of her days she never ceased to marvel at it.

Finding friends in Savannah whenever he could go there, and occupying his leisure hours in letter-writing and in sketching,​23 Lee passed the winter of 1829‑30. Such social life as he could have in Savannah must have been less pleasant than it would normally have been to a young man of his temperament because the proud name of the Lee family had become involved in a humiliating public scandal in the very circles where it had stood highest. In 1817, Henry Lee, Robert's half-brother, son of "Light-Horse Harry" by his union to Matilda Lee of Stratford, had married a young woman of means in Westmoreland County.​24 Living as a country gentleman, first at Stratford and then at Fredericksburg, Major Lee had dabbled in letters, much to the neglect of his estate, and had served as assistant postmaster general under J. Q. Adams.​25 In 1827, Henry Lee's affairs had become so much involved that a judgment of $9000 was procured against him by Henry Storke. As Lee could not meet this, Stratford had been sold for $11,000 and, on June 30, 1828, had formally passed out of the Lee family.​26 Impoverished and embittered, Henry Lee had tried to make a living by writing. By inheritance he was a Federalist, but he had become a protagonist of Andrew Jackson. He had resided at "The Hermitage" after the sale of Stratford, had been engaged in arranging Jackson's military papers,​27 and had written several polemic in behalf of "Old Hickory." Jackson found these last to be indited in a temper that matched his own and he felt much gratitude to Lee. When he became President, he named his defender United States consul to Morocco. It was a vacation appointment, which Lee was very glad to accept. He left the country for his post, only to find that he left a storm behind him. His wife had a younger sister, co-heiress to her father's estate. In some  p98  way, Henry Lee became enamoured of her and had been guilty of misconduct with her. The ugly facts apparently had been whispered about, and perhaps had caused Henry Lee to be socially ostracized, but they had led to no public reprisals. Now, when Jackson submitted his name for confirmation by the Senate on February 3, 1830,​28 an open fight was made on him. As he had already admitted some items of the charge, no defense was made. Every senator who cast a ballot voted against him, among them his long-time friend and college-mate, John Tyler.​29 The whole of the scandal became common knowledge in March, 1830. Henry Lee had to leave his post, and after a stay in Italy, removed to Paris, where he was to live until his death, seven years later.30

This affair must have been an intense humiliation to Lieutenant Lee. Much as he had cherished the memory of his father, he could not have been ignorant of "Light-Horse Harry's" financial reputation, and now to have his father's name disgraced by the son who bore it was to add the blush of shame to the ruddy complexion of the young engineer. So far as is known, he never referred in later life to his half-brother, though he possessed and doubtless studied the one volume that "Black Harry" issued in 1837 of a projected life of Napoleon.​31 Significantly, Robert Lee failed, in later years, to name any one of his three sons Henry, perhaps in the belief that to do so would be to revive the scandal. Doubtless as he read by the candle of his crude quarters on Cockspur Island the story of his brother's misdeeds, he was strengthened in his resolution to efface by his own conduct the blot on the proud scutcheon of the Lees. Such things in a man's life are not to be proved by citation or confirmed by footnotes, but there is every reason to believe that the stern morality of Robert Lee was stiffened by the warning of his brother's fall. In exactly the same way, the rigid exactness of the son in all money-matters, small and large, was a reaction from his father's laxness.

---

The Author's Notes:

1 Fitz Lee: General Lee (cited hereafter as Fitz Lee), 17.

2 Mason, 24‑25. This incident may have occurred during Robert's furlough of 1827.

3 G. C. Lee to Hill Carter, MS., March 11, 1829; Carter MSS.

4 Ann Carter Lee to Smith Lee, MS., April 10, [1827] Lee MSS., infra.

5 An anonymous cousin, quoted in Long, 26.

6 R. E. Lee, Jr. 363; see infra,over­looked them.',WIDTH,225)" onMouseOut="nd();">º p89.

7 Report of the Committee Appointed . . . for the Purpose of Reinterring the Remains of General Henry Lee . . . at Lexington, Virginia (Richmond, Va. [1913]).

8 Lee MSS., placed at the writer's disposal by the late Mrs. C. P. Cardwell, acting for all the generous descendants of Captain Smith Lee.

9 Lee MSS., loc. cit.

10 A dress of Ann Lee, preserved in Richmond, would indicate that she was a woman of •about 5 feet 6 inches and of middle weight.

11 His first official letter to the chief of engineers, Charles Gratiot, notifying that officer where he was in case he was called to duty before the expiration of his furlough, was written from Georgetown July 31, 1829; U. S. War Dept. MSS. Engineers, File 137. These MSS. are the letters from officers of engineers, and are filed by date and by the letter of the sender's last name. Those of Lee, therefore, appear under the letter L, and are cited hereafter as Eng. MSS., with the date and file number.

12 Will of Ann H. Lee, Will Book 1, P., Fairfax County, Va. The bond, $60,000, was, under Virginia law, twice the estimated value of the estate.

13 Long, 30.

14 Brock, 155 ff. Mrs. Washington, it will be remembered, was born Martha Dandridge and had first married John Daniel Parke Custis, by whom she had four children, among them John Parke Custis. In 1773, while still a very young man, John Parke Custis married Eleanor Calvert of Mount Airy, Prince George County, Md., a granddaughter of the sixth Lord Baltimore. George Washington Parke Custis was their son. In 1781 John Parke Custis died, presumably of the hardships sustained during the campaign of that year, when he served as aide to Washington. His daughter Nellie and the boy then came to Mount Vernon to reside.

15 Cf. Long, 30. For the kinship of the two families and for the ancestry of the Fitzhughs, see E. J. Lee, 82, 89.

16 Charles Carter Lee to Hill Carter, MS., March 11, 1829; Carter MSS.

17 Photostat, Virginia State Library, Records Adjutant General's Office, U. S. Army, Adjutant General's Office, General orders No. 50 and orders of Aug. 5, 1829; Lee to Gratiot, Eng. MSS., 140; R. E. Lee, Jr. (2d ed.), 446.

18 Nat died at Cockspur Island or nearby. The date is not known (Mason, 23).

19 1 Cullum, 425.

20 Cf. E. J. Lee, 101.

21 Samuel Babcock to George Gibson, commissary-general, Feb. 1, 1830; R. E. Lee, Jr. (2d ed.), 447.

22 The defenses do not appear officially as "Fort Pulaski" until 1833. (Cf. report of the chief engineer of the army, 1832‑33; Ex. Docs., 1st sess., 23d Cong., vol. 1, p53.)

23 Two of his sketches, the only examples known to be in existence, are reproduced in R. E. Lee, Jr. (2d ed.), 449. Lee gave them to Miss Sarah Anna Minis of Savannah.

24 E. J. Lee, 403.

25 J. S. Bassett: Correspondence of Andrew Jackson, 3, 291.

26 Westmoreland County MS. Deed Book 26, p78. The next conveyance of Stratford, Dec. 13, 1843, was from Henry Storke to his widow, Elizabeth (Ibid., Deed Book 31, p473), by whom it was bequeathed, in 1865, to her grandnephews, Charles E. and Richard H. Stuart (Ibid., Deed Book 83, p412). On Nov. 5, 1919, Richard H. Stuart deeded the property to Charles E. Stuart (Ibid., Deed Book 83, p500), from whom it was purchased by the Lee Memorial Foundation.

27 Henry A. Wise met him there, Oct. 9, 1828; B. H. Wise: Life of Henry A. Wise, 28.

28 Journal of the United States Senate, 1830, Appendix, 408.

29 Journal of the United States Senate, 1830, Appendix, 423; Basset, op. cit., 3, 291.

30 James Parton: Life of Andrew Jackson, vol. 3, pp274 and 297‑98, quoting letter of March 11, 1830, from Major W. B. Lewis to Colonel L. C. Stanbaugh. There is in the Library of Congress a MS. letter, Aug. 24, 1833, from Henry Lee to Richard T. Brown, confessing his misdeed but arguing, in a singularly callous strain, that his conduct was no worse than that of Thomas Jefferson, whom he accused of attempting to betray the wife of a friend.

31 H. Lee, The Life of Napoleon Bonaparte Down to the Peace of Tolentino and the Close of the First Campaign in Italy (London and Paris, 1837).

Images with borders lead to more information. The thicker the border, the more information. (Details here.)
A table of bibliographic abbreviations can be consulted here and some will find the index useful: A-B • C • D-F • G-H • I-K • L • Robert E. Lee • M • N-P • Q-R • S-Z
UP TO:		Robert E. Lee homepage	American & Military History	Home
OFFSITE:		Fort Pulaski: Robert E. Lee	Cockspur Island: Fort Pulaski	(The two U. S. Park Service sites are not linked to each other)
A page or image on this site is in the public domain ONLY if its URL has a total of one *asterisk. If the URL has two **asterisks, the item is copyright someone else, and used by permission or fair use. If the URL has none the item is © Bill Thayer. See my copyright page for details and contact information.