I stumbled on this New York Times article that describes just how Manhattan centric the origins of the Manhattan Project actually were.
Manhattan was central... because it had everything: lots of military units, piers for the import of precious ores, top physicists who had fled Europe and ranks of workers eager to aid the war effort. It even had spies who managed to steal some of the project’s top secrets... The borough had at least 10 sites, all but one still standing. They include warehouses that held uranium, laboratories that split the atom, and the project’s first headquarters — a skyscraper hidden in plain sight right across from City Hall.
Imagine warehouses full of uranium in Manhattan today! It shows you just how different 1940's Manhattan was. Did you know that J. Robert Oppenheimer grew up on Riverside Drive and went to the Ethical Culture School? The Times also includes an interactive map showing some of the major sites of the Manhattan project.