In 1885, the Town of Tulare became the first institution of higher learning in the Southern and Central San Joaquin Valley. Tulare's Smith College was affiliated with the University of Southern California and was one of the first colleges to be opened due to USC's expansion program. They built Smith College in Smith's Park, a triangular piece of land at Park Drive and Elm Avenue near "D" Street on Tulare's Westside.
Dr. William Theodore Felinghuyvon Smith was the founder and financial supporter of the College, which bore his name. Dr. Smith was a Civil War Veteran from Illinois, who was a practicing physician and a teacher. He acquired extensive landholdings in Tulare and generously donated land for the College. He also erected a white three-story building that housed the College and a dormitory. In 1885, the Town of Tulare was a prosperous community and home to the only Southern Pacific Railroad Stop between Mojave and Stockton. The town and the College continued to prosper until disaster struck in the 1890s. Drought, depression, and the relocation of the Southern Pacific Railroad's Stop to Bakersfield almost destroyed the Town of Tulare and forced the closure of the College.
Eventually, the building was sold to James Fleener and moved to Armona, used as a hotel. In 1937, Mrs. James E. Anderson, wife of Smith College's former commercial and mathematics teacher, related the College's history for an article in the Advance-Register as follows: "Most of the girl students roomed and boarded in the dormitory, located on the third floor. The first floor had two classrooms, the commissary department, and the dining room. The second floor contained the chapel seating fifty people and classrooms. Verandas extended all around both second and third floors, with French doors opening from all rooms onto them. The view in all directions from the upper veranda was glorious, the Sierra, the Coast Range, the vast valley, and the then great Tulare Lake."