Sloan was unable to significantly improve production despite a workforce that grew to approximately 125 by January 1863. The increased number of workers, however, was offset by a high level of turnover among the personnel. Consequently, many workers failed to gain the expertise and experience essential for learning the complex production process. Sloan soon resorted to using enslaved African American workers, who were prohibited from leaving the premises and were accustomed to skilled work. Even with more reliable labor, however, arms production did not improve, in part because of raids by native Unionists and the Federal army located in Knoxville, Tenn.
An angry Gorgas replaced Sloan with Capt. C. C. McPhail, a stern disciplinarian. Before McPhail's arrival had any significant impact, Federal raids intensified and the decision was made to transfer the armory equipment to Columbia, S.C. Only 900 rifles were ever manufactured at the Asheville works, and extant examples are rare.