At some points this became a positive embarrassment to Union commanders. A few days after General Butler took command of the Union troops at Fortress Monroe in May, 1861, the agent of a rebel master came to insist on the return of three slaves, demanding them under the fugitive-slave law. Butler replied that since their master claimed Virginia to be a foreign country and no longer a part of the United States, he could not at the same time claim that the fugitive slave law was in force, and that his slaves would not be given up unless he returned and took the oath of allegiance to the United States. In reporting this, a newspaper pointed out that as the breastworks and batteries which had risen so rapidly for Confederate defense were built by slave labor, negroes were undoubtedly "contraband of war," like powder and shot, and other military supplies, and should no more be given back to the rebels than so many cannon or guns. The idea was so pertinent, and the justice of it so plain that the name "contraband" sprang at once into use. But while this happy explanation had more convincing effect on popular thought than a volume of discussion, it did not solve the whole question. By the end of July General Butler had on his hands 900 "contrabands," men, women and children of all ages, and he wrote to inquire what was their real condition. Were they slaves or free? Could they be considered fugitive slaves when their masters had run away and left them? How should they be disposed of? It was a knotty problem, and upon its solution might depend the loyalty or secession of the border slave States of Maryland, West Virginia, Kentucky and Missouri, which, up to that time, had not decided whether to remain in the Union or to cast their fortunes with the South.