electronic configuration - Why does the energy gap for π - π* transitions shrink with the size of the pi-conjugated system?

Quoting from this site:

As conjugated pi systems become larger, the energy gap for a π - π* transition becomes increasingly narrow, and the wavelength of light absorbed correspondingly becomes longer. The absorbance due to the π - π* transition in 1,3,5-hexatriene, for example, occurs at 258 nm, corresponding to a ΔE of 111 kcal/mol.

It's also quite easy to see that this is true glancing at various dye absorption tables.

But why is this true? I can understand why the optical cross-section would increase, implying a higher quantum efficiency at some maximum absorbance wavelength (is this generally true?), but why would the energy gap between the ground state and the excited state correspondingly shrink?

Answer

Klaus Warzecha's answer pretty much answers your question. But I know that this subject is easier to understand if supported by some pictures. That's why I will take the same route as Klaus at explaining the concept behind why the absorption in conjugated systems is shifted to higher wavelengths but I will provide some pictures on the way.

In a conjugated carbon chain or ring system you can think of the $\ce{C}$ atoms as $\text{sp}^{2}$-hybridized. So, each carbon has 3 $\text{sp}^{2}$ orbitals which it uses to form $\sigma$ bonds and 1 $\text{p}$ orbital which is used to form $\pi$ bonds. It is the $\text{p}$ orbitals that are responsible for the conjugation and their combinations according to the LCAO model are the interesting part since the HOMO and LUMO of the system will be among the molecular orbitals formed from the conjugated $\text{p}$ orbitals.

For a start take ethene, the simplest $\pi$-system, being comprised of only 2 carbon atoms. When you combine two atomic orbitals you get two molecular orbitals. These result from combining the $\text{p}$ orbitals either in-phase or out-of-phase. The in-phase combination is lower in energy than the original $\text{p}$ orbitals and the out-of-phase combination is higher in energy than the original $\text{p}$ orbitals. The in-phase combination accounts for the bonding molecular orbital ($\pi$), whilst the out-of-phase combination accounts for the antibonding molecular orbital ($\pi^{*}$).

Now, what happens when you lengthen the conjugated system by combining two ethene fragments? You get to butadiene. Butadiene has two $\pi$ bonds and so four electrons in the $\pi$ system. Which molecular orbitals are these electrons in? Since each molecular orbital can hold two electrons, only the two molecular orbitals lowest in energy are filled. Let's have a closer look at these orbitals. In $\Psi_1$, the lowest-energy bonding orbital, the electrons are spread out over all four carbon atoms (above and below the plane) in one continuous orbital. There is bonding between all the atoms. The other two electrons are in $\Psi_2$. This orbital has bonding interactions between carbon atoms 1 and 2, and also between 3 and 4 but an antibonding interaction between carbons 2 and 3. Overall, in both the occupied $\pi$ orbitals there are electrons between carbons 1 and 2 and between 3 and 4, but the antibonding interaction between carbons 2 and 3 in $\Psi_2$ partially cancels out the bonding interaction in $\Psi_1$. This explains why all the bonds in butadiene are not the same and why the middle bond is more like a single bond while the end bonds are double bonds. If we look closely at the coefficients on each atom in orbitals $\Psi_1$ and $\Psi_2$, it can be seen that the bonding interaction between the central carbon atoms in $\Psi_1$ is greater than the antibonding one in $\Psi_2$. Thus butadiene does have some double bond character between carbons 2 and 3, which explains why there is the slight barrier to rotation about this bond.

You can construct the molecular orbitals of butadiene by combining the molecular orbitals of the two ethene fragments in-phase and out-of-phase.

This method of construction also shows why the HOMO-LUMO gap of butadiene is smaller than that of ethene. The molecular orbital $\Psi_2$, which is the HOMO of butadiene, is the out-of-phase combination of two ethene $\pi$ orbitals, which are the HOMO of ethene. Thus, the HOMO of butadiene is higher in energy than the HOMO of ethene. Furthermore, the molecular orbital $\Psi_3$, which is the LUMO of butadiene, is the in-phase combination of two ethene $\pi^{*}$ orbitals, which are the LUMO of ethene. Thus, the LUMO of butadiene is lower in energy than the LUMO of ethene. It follows that the HOMO-LUMO energy gap is smaller in butadiene than in ethene and thus butadiene absorbs light with longer wavelenghts than ethene.

If you continue to lengthen the $\pi$ system by adding more ethene fragments you will see that the HOMO and LUMO are getting closer and closer together the longer the $\pi$ system becomes.