Upon arriving to Huong Tich grotto, there is a 120 steps descend as the temple lies deep within the grotto. At the entrance to the grotto, there is "heaven's gate" (path to heaven) and "hell's gate" (descend to hell).

In the 18th century, Lord Trinh Sam had 5 words, "Nam Thien De Nhat Dong" meaning, "the most beautiful grotto under southern sky", carved onto the granite slab at the entrance to Huong Tich grotto. According to legend, the cave was discovered two thousand years ago. However, local people only began to place a shrine there in 1575, and later a larger temple was built.

There are many colorful stalagmites and stalactites within the cave. Each is given a name for their purported features. Some of the more famous ones are Cay Gao - Rice Stick, Cay Vang - Golden stick, Nui Co - the maiden and Nui Cau - the youth. Nui Co and Nui Cau supposedly look like the head of a young children. It is believed that couples who wish for children often pay homage to Huong Tich grotto and to especially visit Nui Co and Nui Cau to pray for their first born.