Quick Answer: In short, we pray both Zuhr and Asr silently because it is the Sunnah of the Prophet (ﷺ) to do so. Certain prayers were read aloud like first two rakats of Fajr, Maghrib and Isha. Others like Zuhr and Asr Salah, the imam or the one who prays alone should recite silently.
Ibadah is worship.
As Muslim’s we need to ask ourselves what is the proper way of Ibadah. How can I fulfill my worship to my Lord.
The simplest answer is obedience. What Allah has told us and what the Prophet had taught us, to do these without question.
There are subjects which the Qur’an has explained to us why things are done a certain way, other times Allah just gives us instructions and we trust him and follow without reason.
There is nothing wrong with pondering, it is human nature to be curious. But do realize wondering why Zuhr and Asr is silent is like wondering why there are only two fardh rakats for Fajr prayer. Some things just are.
The Sunnah of the Prophet To Pray Silently
Hazrat Malik (R.A) has said that I accompanied a delegation to the service of the Holy Prophet (Peace be upon Him). We stayed there for 20 days and nights and learned Islam from the Holy Prophet. Prophet Muhammad (ﷺ) advised them to,
“Pray as you have seen me praying” 
He then tells them to go back home and teach others just as you observed. The Prophet (ﷺ) had taught us in detail how to pray Salah.
In another hadith, it was narrated by Abu huraira that Qur’an is to be recited in every prayer and certain prayers are read aloud while others were prayed by the Prophet (ﷺ) silently so we recite them silently as well. 
To Avoid Persecution
The above is more than enough reason to pray Zuhr and Asr silently. There are some scholars which tried to understand this subject deeper which will expand on here.
The wisdom behind Zuhr and Asr being silent can be understood by studying the situation of early Muslims in Mecca. During this time period, Muslims and the Prophet were subjected to persecution. Thus the Salah which was offered in daylight (Zuhr and Asr) were read silently to evade disturbance of their prayer and further persecution.
After sunset, the Quraish were busy with their social gatherings or personal celebrations where they used to drink. During this time it was more suitable for Muslims to make their prayers out loud.
So Zuhr and Asr were prayed mute to avoid harm.
Now, with regards to a ruling which is established in shariah, if the reason no longer applies, as long as the ruling is in accordance with the fitrah (or if it remains as one of the symbols in Islam) then it should be continued even if the original reason is no longer applicable.
Whether the above explanation is true or not shouldn’t really be of concern. Allah knows best. The way the Prophet prayed is known and described and we can follow this way without questioning. In the Qur’an the only mention of the manner in which we should recite the prayer was mentioned in Surah Al-Isra ayat 110:
“And do not recite [too] loudly in your prayer or [too] quietly but seek between that an [intermediate] way”.
 We came to the Prophet (ﷺ) and stayed with him for twenty days and nights. We were all young and of about the same age. The Prophet (ﷺ) was very kind and merciful. When he realized our longing for our families, he asked about our homes and the people there and we told him. Then he asked us to go back to our families and stay with them and teach them (the religion) and to order them to do good things. He also mentioned some other things which I have (remembered or [??] ) forgotten. The Prophet (ﷺ) then added, “Pray as you have seen me praying and when it is the time for the prayer one of you should pronounce the Adhan and the oldest of you should lead the prayer. Sahih Al-Bukhari
 Narrated Abu Huraira: The Qur’an is recited in every prayer and in those prayers in which Allah’s Messenger (ﷺ) recited aloud for us, we recite aloud in the same prayers for you; and the prayers in which the Prophet (ﷺ) recited quietly, we recite quietly. If you recite “Al-Fatiha” only it is sufficient but if you recite something else in addition, it is better. Sahih Al Bukhari