This is the best place for the history enthusiast. Check out underground caverns with guided tours from Plain Road in the Golden Bay.
It made things much easier if you speak their languages, for this reason a personal assisted Taiwan free-and-easy trip is generally recommended.