Below is an excerpt from an essay by Prof. Lokesh Chandra, which reveals the raison d’etre of the caves, cave-dwelling, and cave temples (2005).
“Caves were the earliest habitations of man. Long after they were not needed for habitation, these caverns drew men to their depths and stirred the spirit as well as the eye. What a dream it was to be in the śunyatā and rūpaṃ of the caves, grooved by the gods, syllogisms in stone, symbolic tangents of what took place in the secret gardens of human ideas and ideals: dharmasya tattvaṃ nihītaṃ guhāyāṃ.
The cave is a womb for illumination, the away from the world to attain transcendence. The Sanskrit word for cave guhā is from the root guh ‘to conceal, keep secret,’ guhya ‘concealed, and secret, mysterious.’
In the Rig-Veda, 10.45.2 guhā (short instrumental case form) means ‘geheim’ (PW): vidma te nāma parāmaṃ guhā yat. The Atharvaveda 11.5.10 guhā nidhi nihītau brāhmaṇasya, which means that the treasures of sacred lore of a brāhmaṇ are hidden in a cave. The Śatapatha-brāhmaṇa 22.214.171.124 equates the guhā with the heart: tasmad idaṃ guheva hṛdayaṃ. In the Śvetāśvatara-upaṇiṣad 3.20 the macrocosmic and microcosmic soul resided in the guhā or deeps of being (anor aniyan mahāw mahiyan ātmā guhāyaṃ).
In a cave, psyche finds the hidden that smiles in its dream of deep meditation. The cave is called a guhā as the categories of knower, knowledge, and knowable are hidden herein, or the soul secrets in it (guḍha jṇātṛ-jṇāna-jṇeya-padārthah asyam, guhāte ‘śyam ātmā īti va’, Vācaspatya lexicon). Bhāgavata-Purāṇa 2.9.24 says: The Divine Being, the Lord of all beings dwells in the guhā (Bhagavān sarva-bhūtānām adhyakṣo ‘vasthito guhām). Brāhmaṇ resides in the guhā, the supreme space (brahma yo veda nihītam guhāyam parame vyoman: lndiche Studien 2.217).
Guhārāja is the best temple-form in Varahamihira’s Bṛhatsamhitā 56.18.25. Guhā is the angelic guardian of a person and hence the name Guhāgupta for a bodhisattva in the Mahāvyutpatti and the Sadharma-puṇḍarika-sūtra. Divinities are located in a cave within a stupa. In sādhana 191 of the Sādhanamālā 2.394 the goddess, Uṣṇiṣa-vijā sits in a cavern in the ćaitya (ćaitya-guhā-garbha-sthitam).
Cave is the solitary vision beyond reflections, where time falls into the timeless. The solemnity of its original silence–deep, dark, oneiric, unfathomable–has so many lessons for meditation. We are hypnotised by solitude, hypnotised by the gaze in a solitary cave. The intimacy of concentration therein leads us to the light on the far horizon. Small caves without murals or relief served as places for austere meditation.
In Tibet, there are caves near monasteries for meditation-retreats. On the other hand, the rock-cut caves with relief and full sculptures served as worship chapels for the laity as well as for the monks.”