hashkafah philosophy - Is it heresy to read the Bible literally?

Rabbi Jonathan Sacks in this debate with Richard Dawkins says:

Rabbis in the 10th century laid down the following principle: If a biblical narrative is incompatible with established scientific fact, it is not to be read literally.... And in Judaism we take a strong view on this, and we have now for 2000 years, and we say reading the Bible literally is heresy.

Is this true? Which rabbis said this, and where exactly? Because while I can understand minor details being understood in the context of scientific knowledge, I'm unfamiliar with a principle that would allow one to throw out whole stories that have long been understood as historical, saying they're just symbolic as Rabbi Sacks does with Adam and Eve. Because with that method, as soon as science contradicts the Bible, then the event didn't happen, and that seems (to me) like taking the easy way and making a slightly different new religion every time just because the old one was proven wrong in a detail.

Answer

The term "literal" and its Hebrew or English counter-parts are not terms with hard and fast meanings so any discussion must necessarily be careful to clarify how the terms are being used in the immediate context.

Generally "literal" meaning refers to the apparent meaning that the reader would reasonably be expected to take away from it. This can also be referred to as the "plain" meaning.

Sometimes "literal" is used in contrast to the "plain" meaning to indicate a meaning that would conform strictly to the words being used without regard to the internal or external context which would provide nuance to how they are used. I would tend to use the term "hyper-literal" for this.

In this later category we may find an excellent example of a "literal" or "hyper-literal" reading which would constitute heresy. Terms such as "the hand of God" or the like do not need to be taken as theological statements about God's anatomy, and those who make such an inference run afoul of one of the Rambam's 13 principles of faith (see also Mishneh Torah Yesodei HaTorah 1:8 and Hilchos Teshuvah 3:7).

Generally, however, we have the concept from Chazal that “אֵין מִקְרָא יוֹצֵא מִידֵי פְּשׁוּטוֹ” Scripture doesn't depart from its plain meaning. (Yevamos 24a and elsewhere). While this is a full discussion on its own, with its own limitations, this is a general principle that guided the Gedolei HaRishonim. Along these lines Rav Saadia Gaon wrote, “it is a well known fact that every statement found in the Bible is to be understood in its literal sense.” (Book of Belief and Opinions/Emunos v'Deos 7:2, Rosenblatt, page 265). And the Rambam explained that, “a mere argument in favour of a certain theory is not sufficient reason for rejecting the literal meaning of a Biblical text.” (Guide for the Perplexed II:25, Friedlander). There is precedent for taking interpretive liberties but one must beware of the limitations of both the Rishonim who speak on the matter as well as basic intellectual honesty.